PrismML's Bonsai 27B squeezes a 27B-class model onto a phone via 1-bit weights

PrismML has released Bonsai 27B, a heavily quantized build of Qwen3.6 27B that shrinks a 27-billion-parameter multimodal model small enough to run on consumer hardware. A standard 27B model needs roughly 54GB at 16-bit precision, and even a 4-bit build runs about 18GB — too big for phones and most laptops. Bonsai ships in two flavors: a ternary variant (weights limited to −1, 0, +1) at 1.71 effective bits and 5.9GB that targets laptop-class quality, and a 1-bit variant (−1, +1) at 1.125 effective bits and 3.9GB that fits within the usable memory budget of an iPhone 17 Pro. The low-bit representation runs end to end across the whole network with no full-precision fallbacks, and both variants keep multimodal vision (via a compact 4-bit vision tower), a 262K-token context, and speculative decoding.

On PrismML’s 15-benchmark suite the ternary build retains 95% of the full-precision baseline and the 1-bit build 90%, with math, coding, and tool-calling — the capabilities agentic workflows lean on — reportedly the least degraded. The company claims the 1-bit model beats the most aggressive conventional low-bit build of the same base while using 2.5x less memory, and cites throughput figures up to 163 tok/s on an RTX 5090 and 87 tok/s on an M5 Max. The pitch is that phone RAM is a harder gate than storage suggests: a 12GB iPhone exposes only ~6GB to an app, shared with KV cache and activations, and Bonsai’s ~4GB footprint is what clears it.

The strategic argument is about where agentic AI runs. Multi-step agents make hundreds of model calls per task, each shipping context, tool calls, and often the user’s private files across the network to a paid cloud API. A capable local model makes the marginal cost of a long agent loop effectively zero and keeps data on-device, enabling offline and privacy-by-construction assistants plus hybrid setups that route only the hardest steps to cloud models. Weights are out today under Apache 2.0 with a free developer-preview API, running on Apple silicon via MLX and NVIDIA GPUs via CUDA. Worth noting the benchmark numbers, memory-density claims, and comparisons all come from PrismML itself and haven’t been independently verified.