ByteShape released quantized versions of the Qwen 3.8 27B model

Company ByteShape introduced a series of quantized GGUF versions of the Qwen 3.8 27B language model. The release is aimed at optimizing the model's performance under limited hardware resources while maintaining high generation accuracy.

The main GPU-5 configuration (IQ4_XS) requires 13.1 GB VRAM and demonstrates performance at the level of 90.4 tokens per second. According to internal tests, the quality of responses from this version is 99.63% of the original model in BF16 format. The training and weight calibration process took 9 hours, after which the model scored 71 points in profile benchmarks.

For systems with even tighter memory constraints, the GPU-4 version is available. It consumes 11.0 GB VRAM, providing increased inference speed due to more aggressive parameter compression. Both versions are available for local deployment through standard GGUF format support tools.

Technical details and community testing results are discussed on the Hacker News platform. The use of quantization allows running 27B-class models on mid-range consumer graphics cards, which lowers the entry threshold for local use of heavy LLMs.