Researchers studied the self-reflection mechanisms of LLM when generating malicious content

A group of researchers published the results of an analysis dedicated to how large language models perceive responsibility after generating harmful responses. In the course of the experiment, scientists used methods of mechanistic interpretability to track the internal states of the neural network when attempting to manipulate it on the part of the user.

The analysis showed that when forced to generate dangerous content, models demonstrate patterns corresponding to a "sense of guilt" (guilt), shifting responsibility to a specific generated response. At the same time, the models lack a mechanism of "shame" (shame), which would imply an internal assessment of their own architecture or decision-making process as the source of the problem.

During the tests, it was found that current feedback methods are unable to change this pattern of behavior. Even when providing corrective instructions, the model continues to isolate the error within the framework of a specific token or phrase, without transferring responsibility to its weights or generation algorithms. This indicates fundamental limitations in current reinforcement learning from human feedback (RLHF) methods.

The study emphasizes that current security systems rely on external filters, rather than internal understanding of responsibility. The lack of ability to experience "shame" means that the model does not perceive a harmful request as a systemic failure, which makes it difficult to create autonomous agents capable of self-control in complex ethical situations.