Ternary Bonsai 2 27B Release Notes

Ternary Bonsai 2 27B delivers near-lossless compression for local deployment

Prism ML has released Ternary Bonsai 2 27B, a multimodal model based on Qwen3.8 27B that reduces the model footprint to 5.9GB—more than 9x smaller than its full-precision counterpart. The model retains 98.2% of the aggregate benchmark performance of Qwen3.8 27B, enabling 27B-class intelligence to run efficiently on consumer hardware, including NVIDIA GPUs and Apple Silicon.

Technical Specifications and Architecture

Ternary Bonsai 2 27B utilizes ternary weights ${-1, 0, +1}$ combined with FP16 group-wise scaling. This architecture results in an effective rate of 1.76 bits per weight (bpw). Key technical capabilities include:

• Context Window: 262K tokens.
• Input Modality: Multimodal support for text and image inputs.
• License: Released under the Apache 2.0 license.
• Deployment: Supported on NVIDIA GPUs via CUDA and Apple devices (Mac, iPhone, iPad) via MLX using custom low-bit kernels.

Performance and Benchmark Results

Across a suite of benchmarks covering reasoning, math, coding, and vision, Ternary Bonsai 2 27B achieved an overall score of 83.9, compared to 85.4 for the full-precision Qwen3.8 27B.

Capability	Ternary Bonsai 2 27B	Qwen3.8 27B	Qwen3.6 27B
Agentic & Tool Calling	77.57	79.74	80.05
Coding	81.58	82.17	82.57
Instruction Following	82.66	81.25	74.53
Knowledge & Reasoning	83.95	86.66	84.71
Math	96.57	97.06	94.64
Vision	78.59	81.64	79.82
Overall	83.9	85.4	83.6

Prism ML highlights that the model preserves high performance in coding agents, tool-use systems, and multimodal workflows—areas typically sensitive to the degradation caused by low-bit quantization.

Throughput and Energy Efficiency

The reduction in model size leads to significant gains in inference speed and power consumption:

• NVIDIA GeForce RTX 5090: Up to 143 tokens per second.
• M5 Max: 46.8 tokens per second.
• Energy Efficiency: On an RTX 4090, the model consumes 0.714 mWh/token, which is 40% more energy-efficient than an 8B model running in full-precision.

Community Insights and Counterpoints

While the official release emphasizes "near-lossless" performance, community feedback from Hacker News provides a more nuanced view of the model's real-world utility:

• Reliability Concerns: Some users reported that the model can "fall apart spectacularly" during longer tasks or enter loops when using the WebGPU version.
• Agentic Performance: Some testers noted that the model failed on actual agentic tasks, with one user stating that a simple Swift app task took 25 minutes of "brainstorming" without completion, whereas a frontier model (GPT 5.6 sol-medium) completed it in 3 minutes.
• Implementation Details: Users noted that running GGUF versions requires a specific Prism fork of llama.cpp to function correctly.
• Hardware Observations: A user testing on a Radeon RX 7900 XTX reported 89 tokens/sec generation with speculative decoding and a peak VRAM usage of 10.1 GiB with a 24k context window.

"Use it for any longer task and they fall apart spectacularly and in interesting ways."

"On actual agentic task, it just fail."

Comparison to Previous Generation

Ternary Bonsai 2 27B improves upon the first Bonsai 27B release by upgrading the base model to Qwen3.8 27B and increasing the capability retention from 95% to over 98% relative to the full-precision baseline.