mozgiDevelopers introduced the compact model BananaMind-2-Pro
The team of developers has released BananaMind-2-Pro — a new language model with a decoder-only architecture. The release is focused on creating efficient solutions for tasks that require high data processing speed with limited computational resources.
The model contains 139 million parameters and supports a context window of size 3,072 tokens. The training process used a dataset of 100 billion tokens. According to the authors, the model demonstrates an increase in performance in ARC and HellaSwag tests compared to the GPT-2 architecture and the previous version of BananaMind-2-Medium.
The technical characteristics allow the model to be deployed on consumer equipment. The training time was 1 hour, during which the model scored 35 points on the internal efficiency metric. The source code and model weights are already available for download in the repository on the Hugging Face platform.
The release of BananaMind-2-Pro expands the line of compact models designed for local inference. Such solutions allow reducing delays when performing basic NLP tasks, eliminating the need to access cloud APIs for simple text operations.
