Bonsai 27B Brings 27‑Billion‑Parameter AI to Phones

Bonsai 27B delivers 27‑Billion‑Parameter AI on a phone

Takeaway: PrismML released Bonsai 27B, a 27B‑parameter model compressed to 3.9 GB (1‑bit) and 5.9 GB (ternary) that runs on consumer devices—including an iPhone 17 Pro—while retaining 90‑95% of full‑precision capability.

Two low‑bit variants enable distinct device classes

Conclusion: The ternary variant (5.9 GB) targets laptops, while the 1‑bit variant (3.9 GB) fits the memory budget of modern smartphones.

Ternary Bonsai 27B uses {‑1, 0, +1} weights with FP16 group‑wise scaling, achieving an effective 1.71 bits per weight. It runs on everyday laptops and provides the highest quality among the two releases.
1‑bit Bonsai 27B uses binary {‑1, +1} weights with the same scaling, achieving 1.125 effective bits per weight. At 3.9 GB it fits within the ~4 GB usable memory on an iPhone 17 Pro, making it the first 27B‑class model that can run on a phone.

Both variants keep the low‑bit representation end‑to‑end across the entire network—embeddings, attention, MLPs, and the language‑model head—without any higher‑precision escape hatches. The vision tower ships in a compact 4‑bit form, enabling multimodal inputs such as screenshots, documents, and camera feeds.

Benchmark performance shows minimal loss of intelligence

Conclusion: Across a 15‑benchmark suite, Bonsai 27B retains 90‑95% of the full‑precision Qwen‑3.6 27B scores, with the smallest drop in math and coding tasks.

Category	Full‑precision Qwen 3.6 27B	Ternary Bonsai 27B	1‑bit Bonsai 27B
Math (GSM8K, MATH‑500, AIME)	95.3	93.4	91.7
Coding (HumanEval+, MBPP+, LiveCodeBench)	88.7	86.0	81.9
Agentic & Tool‑calling (BFCL v3, TauBench)	80.0	74.0	66.0
Instruction following (IFEval, IFBench)	78.4	71.8	65.8
Knowledge / STEM (MMLU‑Redux, MuSR)	83.1	77.0	73.4
Vision (MMMU Pro, OCRBench)	72.6	65.2	59.6
Overall	85.0	80.5	76.1

The table shows that math and coding scores drop by less than 2 points, while tool‑calling—critical for agentic workloads—remains within a few points of the baseline. The authors note that a conventional 4‑bit quantization of the same base model occupies 2.5× more memory and scores significantly lower than the 1‑bit Bonsai variant.

Intelligence density: more capability per gigabyte

Conclusion: Bonsai 27B achieves an intelligence‑density of 0.53 points / GB, over 10× the full‑precision baseline and ~2.7× the best competing low‑bit model. The authors introduce intelligence density (performance per GB of model size) as a metric to compare compression approaches. Figure II in the announcement illustrates that Bonsai 27B’s 0.53 points / GB surpasses all other models in the same parameter class, confirming a dramatic Pareto shift.

Why on‑device execution matters for agentic AI

Conclusion: Local models eliminate per‑token network costs, protect user data, and enable offline, persistent assistants. Agentic workloads involve hundreds of model calls per task, each carrying context and producing structured outputs. Running such loops in the cloud incurs latency, token‑based pricing, and data‑privacy concerns. By fitting a 27B‑class model on‑device, developers can:

Deploy zero‑marginal‑cost, multi‑step agents that run entirely offline.
Keep private data (documents, screenshots, personal files) on the device.
Combine local and cloud models in a hybrid architecture, routing privacy‑sensitive steps to the device and reserving cloud resources for the hardest inference.

The announcement includes a demo (Carousel I) of an end‑to‑end agentic workflow powered by the ternary variant on an RTX 5090, and a second demo (Demo II) showing multimodal agentic use‑cases on an iPhone 17 Pro Max.

Real‑world performance numbers

Conclusion: On high‑end GPUs, Bonsai 27B reaches >160 tokens/s (1‑bit) and >130 tokens/s (ternary); on Apple M5 Max it reaches 87 tokens/s (1‑bit) and 58 tokens/s (ternary). These throughput figures demonstrate that the compressed models are not only memory‑efficient but also fast enough for interactive applications when paired with modern hardware.

Community reactions and practical concerns

Conclusion: Early adopters praise the breakthrough but raise questions about comparison baselines, CPU performance, and integration tooling.

Comparison requests: Users ask how Bonsai 27B stacks up against Google’s 4‑bit QAT Gemma 4 12B model, which occupies ~7 GB and shows strong tool‑use and vision capabilities.
CPU inference: A community member reported 9 tokens/s prompt and 6 tokens/s generation on a Ryzen 7 5700X for the binary variant, noting that ternary inference is still slower on CPUs.
Tool‑calling drop: Some commenters point out that a 5‑point loss in tool‑calling may be significant for real‑world agents.
Deployment hurdles: Users mention that the models appear on Hugging Face but may not work out‑of‑the‑box with LM Studio or llama.cpp without updated kernels.
Future scaling: Questions arise about whether larger models could be compressed to fit a 16 GB GPU, indicating interest in further size reductions.

Licensing and availability

Conclusion: Bonsai 27B is released under the Apache 2.0 license, with weights and low‑bit kernels publicly available. Developers can download the model from the PrismML repository, use the provided MLX (Apple) and CUDA (NVIDIA) runtimes, and access a limited‑time preview API for rapid prototyping.

Outlook: intelligence density as a new AI progress axis

Conclusion: The authors argue that raw capability determines what a model can do, while density determines where it can run; shifting the density frontier expands AI’s reach to billions of devices. PrismML plans to apply the same compression methodology to larger models and emerging architectures, suggesting that future releases will continue to push capability into ever smaller footprints.

Bottom line: Bonsai 27B proves that a 27‑billion‑parameter, multimodal, agentic‑capable LLM can be compressed to under 4 GB, run on a modern smartphone, and retain the majority of its original performance—opening the door to on‑device AI assistants, privacy‑preserving workflows, and hybrid cloud‑edge deployments.