SpeckLabs introduced the compact Speck2-140M model

Company Speck2-140M released a new base language model focused on efficient work with the English language. The system architecture includes 140 million parameters, making it suitable for running on devices with limited computational resources.

The model is based on a combination of grouped-query attention and gated convolution mechanisms. Training was conducted on a corpus of 20 billion tokens. Developers confirmed the correct operation of the system when using the greedy decoding method, which ensures predictability of text generation.

The model is available for integration through the Transformers library. To load the weights in the code, you need to use the trust_remote_code=True parameter, since the architecture requires specific settings for the correct initialization of convolution and attention layers.

The release of such compact solutions allows for the integration of language capabilities directly into local applications without the need to access cloud APIs. The low number of parameters ensures high inference speed, which is critical for tasks that require minimal delay when processing text queries.