Researchers introduced the CRN v2 module for error correction in LLM

A group of researchers published a work Safe Error Correction for Language Models, in which they presented the CRN v2 module for correcting errors in the responses of frozen language models. The system allows for correcting logits without the need for retraining the base architecture, which prevents the loss of the model's original skills. The CRN v2 module contains 34 million trainable parameters, which is only 0.73% of the volume of the base Gemma 4 E2B model (4.65 billion parameters). Training was conducted on a dataset of 83,400 data pairs using the DPO (Direct Preference Optimization) method. During tests on the CEHRI exam, the system corrected 53.3% of the base model's errors, while preserving performance on the MMLU and BoolQ benchmarks. The authors compared the solution to the LoRA method, which with a similar parameter budget (6.6 million) demonstrates higher correction accuracy (83.3%), but leads to a degradation of the model's basic capabilities by 30–75%. The key factor in the success of CRN v2 was the use of the KL-divergence coefficient (lambda=0.1), which keeps the model from deviating from the original weights when making corrections. The study confirms that using a frozen base in combination with an external correction layer is an effective way to improve the reliability of LLM. The source code and model weights are available under the CC BY 4.0 license for further testing in applied tasks.