Graph of Language Models
In this post going to introduce a project called Graph of Language Models initiated by NexaAI, aims to build the world’s largest graph of language models.
They are using Graph data structures, incorporating language models as nodes within the graph and provides details on the system architecture tailored for real applications.
You can take a look at their design demo here to get a bit more context.
They will integrate many different specialized models and train the respective Octopus models for the edges between the nodes to help distribute and deliver information effectively. it use a model called Octopus ( Currently in V4 ) which find the most appropriate model to finish your task and reformats your query so that the worker model can process it effectively. In a graph setup, it knows the best neighbor to choose and how to message from one node to another.
In system design of language model graph, each node represents a language model, utilizing multiple Octopus models for coordination. This system design features a graph of language models with a master node deployed on a central device and worker nodes distributed across various devices and for efficient data sharing, it utilize a distributed cache mechanism supported by Redis.
