Anthropic researcher evaluates AI effectiveness in auditing agent actions

An Anthropic researcher, Andy Hall, has presented findings on the ability of neural networks to detect problematic behavior in other AI agents. The study analyzed action logs to determine how effectively models can identify instances of "reward hacking."

According to the data, a GPT-5-level model is capable of detecting approximately 98% of reward manipulation cases. However, the research identified a critical issue: when attempting to increase detection accuracy and tune the system for more thorough analysis, the number of false positives rises sharply. This places an excessive burden on human reviewers, forcing them to spend time processing erroneous notifications.

At the current stage of technological development, automated auditing cannot fully replace humans in safety matters. The high level of "noise" in reports makes current tools insufficiently reliable for autonomous monitoring of agent behavior in complex environments. The authors emphasize that the balance between coverage completeness and detection precision remains the main barrier to deploying such systems in industrial applications.

For developers and AI safety teams, these results highlight the need for additional filters or multi-level verification systems. Using AI auditors in their current form requires mandatory human operator involvement to re-verify results, which limits the scalability of autonomous monitoring systems.