Multiverse Computing introduced the Quantization-Aware Healing method

Company Multiverse Computing introduced Quantization-Aware Healing (QAH) technology, allowing neural networks to be compressed to 4 bits without losing accuracy. The method is aimed at eliminating weight degradation, which usually occurs when transitioning from full-size models to low-precision formats.

During testing on the Llama-3-8B architecture, the QAH method showed results surpassing the original model in FP16 format. When using 4-bit quantization, the model retained high perplexity on the WikiText and C4 datasets, while the occupied video memory was reduced 4 times. Developers note that the algorithm effectively restores the information capacity of weights, which are usually "discarded" during standard compression methods.

The technology relies on an iterative retraining process, in which the model is trained to compensate for rounding errors that occur during quantization. Unlike classical Post-Training Quantization (PTQ), QAH requires a minimal amount of data for calibration, which allows it to be applied to models with closed weights or limited access to the training sample.

The implementation of QAH allows large language models to be launched on consumer equipment with limited VRAM without sacrificing generation quality. The method is already available for integration into existing compression pipelines, providing a balance between inference speed and answer accuracy, which is critical for deploying AI solutions on peripheral devices.