Bonsai 27B: A 1-Bit LLM That Fits on a Phone

PrismML released Bonsai 27B on July 14, and its 1-bit version is the first 27B-class model that fits and runs on a phone, packing into roughly 3.9 GB of memory. The announcement went live hours ago, and the claim that matters is simple: a model in the same weight class as the flagship open models from Google and Alibaba now sits on an iPhone 17 Pro without offloading to the cloud. Our read: the raw benchmark scores are not the story. The story is that native 1-bit training just turned a 54 GB model into something that lives in your pocket while keeping about 90 percent of its intelligence.

• Bonsai 27B ships in two flavors: a Ternary build at 1.71 bits per weight (5.9 GB, for laptops) and a 1-bit build at 1.125 bits per weight (3.9 GB, for phones), both under an Apache 2.0 license.
• The weights are trained natively at 1-bit, not quantized after the fact, so every weight is -1 or +1 across the embeddings, attention layers, and language-model head.
• On PrismML's 15-benchmark suite the 1-bit build scores 76.1 overall versus 85.0 for the full-precision Qwen 3.6 27B baseline, retaining roughly 90 percent of the quality at about one-tenth the memory.
• It is multimodal, not a text-only toy: 262K-token context, vision, tool calls, and agentic computer-use loops, running up to 163 tokens per second on an RTX 5090 and 87 on an M5 Max.

What did PrismML actually release?

On July 14 PrismML, a startup founded by Caltech researchers, released Bonsai 27B, the flagship of its Bonsai family that already included 8B, 4B, 1.7B, and an image model. It is based on Qwen3.6 27B and comes in two compressed variants under the permissive Apache 2.0 license. The Ternary build uses three-state {-1, 0, +1} weights with FP16 group scaling at 1.71 effective bits per weight and 5.9 GB, aimed at everyday laptops. The 1-bit build uses pure {-1, +1} weights at 1.125 bits and 3.9 GB, small enough for a 12 GB iPhone 17 Pro where roughly 6 GB is usable to apps. For reference, the same model at 16-bit precision is about 54 GB, and even standard 4-bit quantization lands near 18 GB, still far too big for a phone. It is available now on Hugging Face and GitHub, through the Locally AI iOS app, and via a Together.ai API with a limited free developer preview.

RelatedAnthropic Scales Project Glasswing to 150 Orgs in 15 Countries

How does a 27B model fit in under 4 GB?

The trick is that Bonsai is trained natively at 1-bit rather than quantized after training, and that is the detail most coverage skips. Standard practice takes a finished 16-bit model and rounds its weights down to 4 or 8 bits, and quality degrades because the network never learned to live with the coarser weights. PrismML instead trains with every weight constrained to -1 or +1 from the start, so the model adapts to the constraint while it learns. This end-to-end approach covers the embeddings, attention layers, and language-model head, not just a handful of layers. It also differs from Microsoft's earlier BitNet work, which used 1.58-bit ternary weights (-1, 0, +1) where the zero lets the network switch a connection off. The 1-bit Bonsai drops the zero entirely, so there is no off switch, which is exactly why the Ternary variant that keeps the zero scores a few points higher. The payoff PrismML highlights is "intelligence density": the 1-bit build delivers 0.53 points of benchmark score per gigabyte, which it claims is more than 10x the full-precision baseline and about 2.7x the best rival low-bit approach.

How good is it, really?

The 1-bit build holds up better than "1-bit" sounds, but it is not free. On PrismML's 15-task suite the full-precision Qwen 3.6 27B baseline scores 85.0, the Ternary build 80.5, and the 1-bit build 76.1. The gaps widen on the hardest categories: coding drops from 88.7 to 81.9 and agentic tool use from 80.0 to 66.0 on the 1-bit build, while math stays strong at 91.7. Vision is the weakest link at 59.6 versus 72.6 for full precision, a reminder that the 4-bit vision tower is carrying real load. The honest framing is that Ternary retains about 95 percent of the baseline and 1-bit about 90 percent, so the 1-bit build is the one you accept a genuine quality hit for in exchange for running on a phone, while Ternary is the closer-to-lossless laptop option.

What does it mean for the market?

The signal for investors is that on-device AI just got materially more credible, and that cuts against the pure cloud-inference thesis. If a 27B-class model runs locally on a phone, some share of queries that would have hit a data-center GPU never leave the handset, which softens the "every token needs cloud compute" narrative and plays to Apple, whose 12 GB iPhone 17 Pro is the reference device and whose MLX framework Bonsai targets. PrismML is private, backed by Khosla Ventures, Cerberus, and Google with continuing support from Samsung, so the direct read-through is to those platform owners rather than a single tradable ticker. Watch memory suppliers too: smaller per-model footprints could temper the on-device DRAM demand story even as cloud HBM stays tight. This is factual analysis, not investment advice.

RelatedClaude Code Adds a Built-In Browser That Reads the Live Web

Our take

The phrase to ignore is "27B on a phone" as a headline, and the number to keep is 90 percent. Fitting a 3.9 GB model in memory is not the same as running it fast, and the experience with earlier Bonsai models on mid-range Android phones was underwhelming even when the weights fit. What makes this release matter is that native 1-bit training is now producing a model that keeps most of a flagship's intelligence at one-tenth the footprint, under a permissive Apache 2.0 license, with vision and tool calls attached. That combination, not the phone demo, is what pressures the assumption that serious models must live in the cloud. If the independent evals hold, the interesting question stops being whether small devices can run big models and becomes how much cloud inference was ever truly necessary.

Original analysis by GenZTech. Reporting informed by PrismML.