Researchers have learned to identify malicious prompts through hidden states of LLM

Researchers presented a method for detecting hazardous content in LLM based on analyzing the neural network's internal activations instead of using external filters. The new approach allows for identifying malicious requests directly during data processing, eliminating the need for additional computational costs for third-party defender models.

As the basis for the experiment, the LLaMA-3.1-8B model was chosen. The authors trained compact classifiers based on MLP, containing only 12.6 million parameters. These probes analyze the model's hidden states, allowing for instant classification of incoming requests as safe or malicious without delays characteristic of traditional security systems.

The effectiveness of the method was confirmed on the WildJailbreak, Beavertails, and AEGIS 2.0 datasets. The results showed F1 metrics of 99%, 83%, and 84% respectively. These indicators are comparable to the quality of work of specialized guard models, which exceed the proposed solution in size by 1000 times.

Using the model's internal states radically reduces delays and resource consumption, which is critical for autonomous systems and devices with limited computational power. This method allows integrating security mechanisms directly into the LLM architecture, providing real-time protection without loss of performance.