Researchers presented the Cache-to-Cache method for direct data transfer between LLMs

A group of researchers presented Cache-to-Cache (C2C) — a new method of direct semantic communication between large language models. The technology allows for the transfer of information directly through the merging of KV caches, bypassing the stage of generating and subsequent parsing of text tokens.

The method is based on the use of a specialized neural network that performs the merging of cache states between models. According to the test results, this approach provides an increase in the accuracy of answers by 6.4–14.2% compared to traditional data exchange in text format. Additionally, the architecture allows for 2.5-fold acceleration of the context transfer process between agents.

Direct work with the KV cache eliminates the overhead of tokenization and detokenization, which is inevitable in standard API interactions. The method is oriented towards scenarios where several models need to exchange complex internal data representations without losing semantic integrity. Detailed technical specifications and benchmark results are published in the scientific paper.

The development opens the way to creating more efficient multi-agent systems, where LLMs can interact at the level of "raw" activation vectors. This allows for reducing delays in distributed AI systems, where the speed of context exchange between information processing nodes is critically important.