The OrcaBonsai-27B-Uncensored model is presented with a method of removing failures

Developers introduced OrcaBonsai-27B-Uncensored — a specialized version of the neural network using the refusal-direction ablation technique in real-time. The model is based on the Ternary-Bonsai-2-27B architecture and optimized for devices with Apple Silicon chips.

The technical implementation of the project relies on the MLX framework and quantization to 2 bits. The key feature is the application of 129 points of intervention in residual connections. This approach allows modifying the model's behavior "on the fly" without the need for retraining or re-quantizing the original weights.

During testing, the system demonstrated a result of 30 points on the internal performance metric, achieving this figure in 3 hours of computation. The use of the intervention method allows bypassing the standard limitations of the model while maintaining the integrity of the base weights, which is critical for operation on consumer hardware with limited memory.

This release expands the capabilities of running heavy models locally on Apple architecture. The use of 2-bit quantization in combination with dynamic intervention in the inference process allows running 27-billion models on devices with less RAM than required for standard FP16 or INT8 versions.