Researchers assessed the ability of LLM to track facts in multi-agent systems

Researchers conducted an experiment simulating a detective investigation involving 8 AI agents. The goal of the study was to test the ability of models to recover key facts from interaction logs under high information load.

During testing, it was found that even the most powerful monitoring systems were unable to extract more than 50% of significant data from the total log array. The results showed a significant gap in efficiency between different architectures when analyzing complex scenarios.

According to the data obtained, the GPT-6 Astra model demonstrated the best accuracy among all tested solutions. At the same time, the Gemini model showed the worst result, significantly inferior to competitors in the ability to structure information and identify logical connections during the simulation.

The research methodology included creating artificial "noise" in the logs to test the models' resistance to irrelevant information. The obtained metrics highlight the current limitations of LLM in long-term contextual analysis and fact verification tasks in multi-agent environments.

The study indicates the need to improve attention and memory mechanisms in modern models. The results emphasize that increasing the number of agents in the system does not guarantee an improvement in analysis quality if the model is not able to effectively filter the data stream.