IFM released a quantized model K2-Horizon-7B-FP8

The team of developers introduced K2-Horizon-7B-FP8 — a new version of the language model with a 7B architecture, optimized for operation in FP8 format. The release is focused on reducing video memory requirements while maintaining the accuracy of calculations characteristic of the BF16 format.

A key feature of the model is support for a contextual window of 512K tokens. The use of FP8 quantization allows for a significant acceleration of the inference process on hardware that supports this data type, while the model retains a dense structure of weights.

In internal tests, the model demonstrated high performance, receiving a score of 10 points on the developers' internal metric in 1 hour of processing time. The model is available for download on the Hugging Face platform and is designed for deployment in environments with limited GPU resources.

The transition to the FP8 format is becoming a standard for optimizing modern LLMs, allowing for the launch of longer contexts on consumer and server equipment without significant loss of generation quality. This release expands the toolkit for developers working with high-load text processing systems.