OpenVINO introduced a quantized version of the Gemma-4-E4B model

Team OpenVINO released an experimental version of the Gemma-4-E4B-it model optimized for local hardware, with model weights compressed to INT4 format using the NNCF library, significantly reducing memory requirements while preserving the basic architecture characteristics.

Conversion to OpenVINO IR format ensures compatibility with 2026.1.0 and later runtime versions, and developers will need Optimum Intel library version 1.27.0 or later for integration into workflows, with the weight preparation and compression process taking around 3 hours, during which the model received a score of 8 points on internal performance metrics.

Using INT4 quantization is a standard approach for running LLM on consumer processors and integrated graphics, and converting the model to OpenVINO format allows for hardware acceleration on x86 and ARM architecture devices, minimizing inference latency, with this release targeting developers creating local AI applications with limited computational resources.

The update expands the ecosystem of tools for deploying Gemma family models in enterprise and user environments, and using optimized formats enables running modern language models on devices that previously did not support working with heavy weights in their original representation.