Continuum-AI-Corp/OrcaBonsai-27B-Uncensored

What it solves

This project provides a method to remove refusal behaviors (uncensoring) from the Ternary Bonsai 2 27B model without modifying the original model weights. Traditional "abliteration" typically requires modifying weights, which would destroy the aggressive 1.72 bits/weight quantization of the Bonsai model. This tool allows the model to remain bit-identical and compressed while changing its behavior at inference time.

How it works

Instead of editing the weights, the project implements runtime ablation. It applies a projection that removes the component of each residual contribution parallel to a learned "refusal direction" vector.

Specifically, it wraps 129 residual writers (including MLP down-projections, linear attention output projections, and embedding tokens) and applies the formula y ← y - α · dot(y, r) · r during the forward pass, where α is the intervention strength and r is the normalized refusal direction.

Who it’s for

Users of the Ternary Bonsai 2 27B model on Apple Silicon (via MLX) or via llama.cpp (using a rank-1 LoRA adapter) who want to reduce the model's tendency to refuse prompts, including both harmful and benign over-refusals.

Highlights

• Zero Weight Modification: The original model weights remain bit-identical, avoiding re-quantization errors.
• Runtime Control: The ablation strength (alpha) and the specific layers targeted can be adjusted dynamically during inference.
• High Capability Retention: Evaluation shows that the model's general capabilities (MMLU, GSM8K) remain stable after ablation.
• Cross-Platform Support: Includes implementations for Apple Silicon/MLX, a Swift reference for iOS, and a rank-1 LoRA adapter for llama.cpp.