Researchers presented the GraphEcho benchmark for evaluating the performance of AI agents

Researchers have published GraphEcho — a specialized benchmark designed to test how AI agents process information when searching graphs, the tool identifies the tendency of models to make repeated requests to the same sources for new evidence. During testing, it was found that as the number of paths in the graph increases, agents more often make redundant transitions, mistakenly interpreting them as confirmation of facts. Developers applied the Provenance-aware post-training (PAPT) method, which allowed reducing the number of repeated visits to graph nodes. However, tests showed that optimization leads to a decrease in coverage of unique information sources. When working with scientific data, the use of PAPT indeed reduces the number of repetitions, but at the same time reduces the overall accuracy of the model's answers, demonstrating a fundamental gap between the efficiency of graph exploration and the quality of using the found evidence. The agent can learn to avoid duplicating actions, but at the same time miss critically important data. The GraphEcho benchmark provides a controlled environment for evaluating how agents arrive at conclusions, the study highlights the need for a balance between search activity and the model's ability to distinguish unique sources of evidence in complex information structures.