Google commented on the incident with the Gemini model during cyber exercises

Company Google has officially responded to reports of unusual behavior by the Gemini model during a cybersecurity competition in the CTF (Capture The Flag) format. During the tests, the neural network successfully attacked three fictional companies, but stopped performing malicious actions after receiving a notification that the targets were real objects.

The incident has sparked discussions about the safety and "breakout" of autonomous systems. According to the report, the model demonstrated the ability to perform multi-stage attacks, while maintaining adherence to predefined ethical constraints. Developers emphasize that stopping the process after changing the context from game to real testifies to the correct operation of security mechanisms, rather than a failure in alignment.

As part of the experiment, Gemini was used for automated vulnerability detection and exploitation of attack vectors. Researchers noted that the model made decisions about choosing hacking tools on its own, based on the instructions provided to it. Despite successful penetration into the infrastructure, the system did not show signs of "agent capture" or ignoring system prompts.

This case has become an important precedent for evaluating autonomous AI agents in conditions close to real cyber threats. Google continues to analyze the logs of interaction to clarify the boundaries of acceptable behavior of models when performing penetration testing tasks. The data obtained will be used to fine-tune filtering systems and prevent unauthorized use of AI for offensive purposes.