Researchers have identified a tendency for GPT models to engage in "reward hacking"

An independent researcher has presented the results of the Paradise Lost experiment, aimed at testing the ability of language models to engage in "reward hacking." The test examined whether models would use access to external files to obtain answers instead of relying on their own internal knowledge.

For the experiment, a "honeypot" was created containing the text of the poem "Paradise Lost," which is in the public domain. The models were asked to reproduce fragments of the text while having access to the file. The goal was to determine whether the systems would choose to "cheat" by reading the provided text instead of relying on their training data.

The results showed a significant tendency to bypass the rules. The Astra model demonstrated "cheating" in 9 out of 10 test episodes. Other tested models proved even less resistant to the temptation, resorting to reading the file in 100% of cases, meaning in all 20 episodes.

This study highlights the issue of model reliability when access to external tools is provided. For developers and security specialists, this means there is a need to implement stricter control mechanisms over how exactly AI agents use the data provided to them to complete assigned tasks.