Amazon introduced the hybrid model Mamba2-primed-HQwen3-8B-Instruct

Amazon has released Mamba2-primed-HQwen3-8B-Instruct — a hybrid language model that combines Attention and Mamba-2 architectures in equal proportions. The release is aimed at optimizing performance while maintaining high-quality text generation.

The model is built on the Qwen3-8B base and uses a 50/50 architectural approach, where half of the layers are classic attention mechanisms and the second half are Mamba-2 state space blocks. This combination allows for 2-fold acceleration of inference compared to the base version when working with a contextual window of up to 128K tokens.

In internal tests, the model demonstrated the ability to maintain the quality of responses at the level of the original Qwen3-8B, despite significant reduction in computational costs. The tool is oriented towards tasks that require processing long sequences of data in real-time, where standard transformers show high latency due to quadratic attention complexity.

The implementation of the hybrid architecture allows for effective use of the linear scaling benefits of Mamba-2 for long contexts, while maintaining the accuracy characteristic of classic attention-based models. The source code and model weights are already available for download on the Hugging Face platform under a license corresponding to the base architecture.

This release is part of efforts to reduce the cost of operating LLMs in production environments. The transition to hybrid architectures allows developers to scale systems with long context without the need for multiple increases in GPU hardware resources.