Researchers presented the Continual Search method for diagnosing errors of AI agents

Researchers have published a scientific paper describing Continual Search — an iterative framework for automatically searching for causes of AI agent failures. The method addresses the inefficiency of LLMs in analyzing long execution logs, where critically important data is often distributed throughout the action trajectory.

In contrast to standard approaches using one-time analysis, Continual Search forcibly directs the model to a sequential search for error evidence. To evaluate its effectiveness, the authors developed the MegaRCA-Mix benchmark, which includes 50 complex scenarios with manual labeling, simulating real tasks with high operation intensity.

Testing results showed a significant increase in the accuracy of attributing causes of failures. When using the GPT-5.5 model, the F1-score metric increased from 0.349 to 0.498, which corresponds to an improvement of more than 40%. The study also confirmed that a quality search strategy allows less powerful models to outperform larger counterparts.

The development is aimed at increasing the reliability of autonomous systems operating in long-term cycles. The use of an iterative approach allows avoiding premature conclusions, to which LLMs are prone when processing large amounts of data. The work is published under the CC BY 4.0 license.