Yandex released the open language model AliceAI-Foundation-80B-A3B-Base

Yandex introduced the basic language model AliceAI-Foundation-80B-A3B-Base, trained from scratch without using third-party weights. The model is distributed under the Apache 2.0 license and is designed to develop agent capabilities in the company's ecosystem.

The model's architecture contains 80 billion parameters, with the number of active parameters being around 11 billion. In internal tests, the novelty surpassed the previous closed version of Alice AI LLM 235B in terms of factual knowledge, mathematical calculations, programming skills, and work with long context. The developers note that the model shows higher results in IMO AnswerBench and LiveCodeBench benchmarks compared to DeepSeek-V4-Flash-Base and Nemotron-3-Super.

The training process took about half a year and included the use of an updated data corpus, redesigned architectural solutions, and optimized hyperparameters. To verify the effectiveness of each stage, the team conducted a series of experiments, training several versions of the model on arrays of 2 trillion tokens each. The full technical report describing the data preparation pipelines and evaluation protocols is published along with the weights.

Along with the model, the company opened two specialized benchmarks — WikiWebFacts and HardMultiQA. These tools are focused on evaluating the quality of answers in the Russian-language context. The publication of these resources is aimed at standardizing approaches to measuring the quality of pre-training, taking into account the complexities of automatic evaluation and the dependence of results on prompts.

The release of AliceAI-Foundation-80B-A3B-Base is part of Yandex's strategy to create a unified reasoning model. The results obtained and the experience gained in the field of scaling laws will be used to further scale the agent functions of Alice.