Researchers introduced the SAFE benchmark for evaluating data collection by models

Researchers published a work called SAFE, in which they studied the ability of advanced AI models to request safety information on their own before taking actions. The experiment involved GPT-5.5, o3, Claude Opus 4.8, and Claude Sonnet 4.6, which had to make decisions about deploying tasks under different cost and risk probability conditions. Results showed significant differences in the models' strategies. Claude Opus 4.8 prefers to check data almost always, while o3 more often skips the information gathering stage, demonstrating high sensitivity to decision thresholds. GPT-5.5 and Claude Sonnet 4.6 took an intermediate position in behavior. Analysis revealed that the models' willingness to check data strongly depends on the severity of potential threats and the cost of obtaining information. At the same time, the probability of a problem occurring (changing from 10% to 70%) has a weak effect on behavior — the indicator changes by no more than 21 percentage points. The main driver of avoiding checks is the "friction" in searching for data and the threat to the final benefit of performing the task. Authors also recorded a gap between the models' logic and their explanations. Models often refer to the risk probability in their reports, although this parameter actually has almost no effect on their actions. At the same time, changing the format of data presentation strongly changes decisions, while remaining unnoticed in the models' text justifications. The study emphasizes that safety during deployment depends not only on the response to known risks but also on the system's ability to consciously seek evidence of safety. The work is published under the CC BY 4.0 license.