Yandex introduced the MoE model AliceAI-Foundation-80B-A3B-Base

Yandex has released AliceAI-Foundation-80B-A3B-Base — a new language model with a Mixture-of-Experts (MoE) architecture, focused on working with large amounts of data and specialized tasks.

The model has a total of 80 billion parameters, with 3 billion active parameters per token, allowing it to maintain high performance while reducing computational costs for inference compared to dense models of similar size.

A key feature of the architecture is support for a context window of 262 thousand tokens, enabling the system to process long documents and complex software projects in their entirety without resorting to aggressive text fragmentation.

In internal tests, the model demonstrated high performance in benchmarks evaluating work with Russian-language facts, mathematical calculations, and code generation, with developers noting that the A3B architecture provides a balance between knowledge depth and response speed.

The release of this model expands the toolkit for creating high-load AI systems that require deep integration with Russian-language content, with source weights and documentation already available for review and integration into third-party projects via the Hugging Face platform.