Technology

Nexa AI introduces Octopus v4: a novel artificial intelligence approach that uses functional tokens to integrate multiple open source models

Following Meta’s release of the Llama3 model and its successor Llama 2 in 2023, there has been rapid growth in the open source landscape for Large Language Models (LLMs). This publication has led to the development of several innovative LLMs. These models have played an important role in this dynamic field by significantly influencing natural language processing (NLP). This paper highlights the most influential open source LLMs such as Mistral’s sparse Mixture of Experts model Mixtral-8x7B, Alibaba Cloud’s multilingual Qwen1.5 series, Abacus AI’s Smaug, and 01.AI’s Yi models, which focus on Focus on data quality.

The emergence of on-device AI models such as LLMs has changed the NLP landscape and offers numerous advantages compared to traditional cloud-based methods. However, the true potential is being realized in combining on-device AI with cloud-based models, leading to a new idea called cloud-on-device collaboration. AI systems can set new standards in performance, scalability and flexibility by combining the power of on-device and cloud-based models. Using both models together allows computing resources to be allocated efficiently: lighter, private tasks are managed by on-device models, and cloud-based models handle heavier or more complex operations.

Nexa AI researchers introduce Octopus v4, a robust approach that leverages functional tokens to integrate multiple open source models, each optimized for specific tasks. Octopus v4 leverages functional tokens to efficiently route user queries to the most appropriate vertical model and optimally adjusts the query format to improve performance. Octopus v4, an updated version of its predecessors – Octopus v1, v2 and v3 models – shows outstanding performance in selection, parameter understanding and query restructuring. It also uses the Octopus model and functional tokens to describe the use of graphs as a flexible data structure that can be efficiently coordinated with various open source models.

In the system architecture of a complex graph in which each node represents a language model and uses multiple Octopus models for coordination, the components of this system are listed below:

  • Deploying worker nodes: Each worker node represents a separate language model. The researchers used a serverless architecture for these nodes and particularly recommended Kubernetes for its robust autoscaling capabilities.
  • Master node deployment: The master node can use a base model with less than 10B parameters. In this article, researchers used a 3B model during the experiment.
  • Communication: Worker and master nodes are distributed across multiple devices, allowing multiple units. Therefore, an internet connection is required to transfer data between nodes.

The thorough evaluation of the Octopus v4 system compares its performance with other useful models using the MMLU benchmark to prove its effectiveness. Two compact LMs are used in this system: the 3B parameter Octopus v4 and another worker language model with up to 8B parameters. An example user query for this model is:

Question: Tell me the result of the derivative of x^3 when x is 2?

Answer: (“Find the derivative of the function f(x) = x^3 at the point where x is 2, and interpret the result in the context of rate of change and tangent slope.”)

Finally, researchers at Nexa AI proposed Octopus v4, a robust approach that leverages functional tokens to integrate multiple open source models, each optimized for specific tasks. Additionally, the performance of the Octopus v4 system is compared with other renowned models using the MMLU benchmark to prove its effectiveness. For future work, the researchers plan to improve this framework by using multiple industry-specific models and incorporating the extended Octopus v4 models with multi-agent capability.


Visit the Paper. All credit for this research goes to the researchers of this project. Also don’t forget to follow us Twitter. Join our… Telegram channel, Discord channelAnd LinkedIn GrOup.

If you like our work, you will love ours Newsletter..

Don’t forget to join our 41k+ ML SubReddit


Sajjad Ansari is a final year student at IIT Kharagpur. As a technology enthusiast, he is concerned with the practical applications of AI, with a focus on understanding the impact of AI technologies and their impact on the real world. His goal is to formulate complex AI concepts clearly and understandably.




Source link