Researchers have proposed the concept of a swarm organization for scaling AI

Researchers have published an analysis of swarm organization, which allows translating the efficiency of computations during execution (test-time compute) from sublinear to superlinear dependence. The method involves using the coordination of multiple agents to solve complex computational tasks.

The approach is based on abandoning monolithic models in favor of distributed systems. As an example, the OpenAI experiment is cited, where a system of 10,000 agents was involved in solving the Navier-Stokes equations. Using such an architecture makes it possible to achieve results that are inaccessible by simply increasing the computing power for one model.

The key factor in efficiency is coordination, where each agent performs a narrowly specialized subtask. This reduces the overhead costs of context processing and allows the system to dynamically redistribute resources depending on the complexity of the current stage of computation. The application of the swarm approach demonstrates the potential to overcome the limitations of traditional model scaling.

This concept changes the approach to designing AI systems, shifting the focus from the size of the model parameters to the organization of interaction between agents. This allows for more efficient use of available hardware resources, increasing the accuracy of problem-solving without the need to retrain basic models.