From 53.80 GB to 5.93 GB, the Deployment Barrier Changes
PrismML’s Ternary Bonsai 2 27B keeps the Qwen3.8 27B architecture and has 27.36 billion parameters, yet reduces the language model weights from 53.80 GB in FP16 to 5.93 GB. According to the release, that makes it runnable on a 16 GB laptop or a single 24 GB GPU. For a model in the 27B class, this is more than file compression: it moves the boundary of local deployment.
The key is to map almost all weights to three values: -1, 0, and +1. Every group of 128 weights shares one FP16 scale, while only a very small set of parameters, including the recurrent state path and normalization weights, remains at higher precision. Those parameters represent about 0.0976% of the total. The ternary values require about 1.585 bits each; after accounting for shared scales, the theoretical storage is about 1.71 bits per weight. The PTQ1_0 packing reaches 1.76 bits per weight and 5.93 GB, while PQ2_0 uses 7.25 GB in exchange for easier unpacking.
Ternary Weights Are Not Simple Rounding
Low-bit representations are more sensitive to outliers, so Bonsai 2 applies a blockwise Hadamard rotation before quantization, using blocks of 1,024. The orthogonal transform preserves total energy while spreading peaks across positions. At runtime, activations receive the matching transform, allowing the ternary approximation to preserve more of the original computation. The rotation is folded into the stored weights and adds no storage bits, although the whitepaper still lists its batch-one overhead as an unresolved cost.
This is why 5.93 GB is not a software-independent number. The model requires PrismML’s llama.cpp fork or the MLX runtime, and the stock llama.cpp cannot load it directly. The custom packing format and specialized kernels have to work together to turn low-bit storage into practical inference speed. Otherwise, some of the memory savings can be offset by unpacking, rotation, and other runtime costs.
The 98.2% Average Hides an Agent-Task Gap
PrismML reports that Bonsai 2 reaches 98.2% of its parent model’s average score across 20 benchmarks. That suggests ternary quantization has not reduced the model to a system suitable only for simple question answering; several capabilities remain close to the original. But the results were measured in thinking mode with high reasoning effort, so the percentage should be read as a result under a particular evaluation setup, not as a guarantee across all workloads.
The gap becomes much clearer when a task requires sustained code changes, tool use, and long intermediate state. On SWE-bench Verified, Bonsai 2 scores 60.8 versus 80.6 for Qwen3.8 27B. On Terminal-Bench 2.1, the comparison is 52.8 versus 69.7, and on LiveCodeBench v6 it is 70.05 versus 90.07. The model therefore looks more like a way to bring broad reasoning capability to local hardware than a drop-in reproduction of the parent model’s long-horizon software engineering reliability.
Once It Fits, Engineering Teams Must Recalculate the Budget
For a deployment team, 5.93 GB answers only whether the language-model weights can be loaded. It does not answer whether a complete application can run reliably. KV cache, activations, and the runtime require additional memory, and a 262K context window is not free. Image input also requires loading a separate vision tower of about 0.63 GB. Being able to start the model on a 24 GB GPU is therefore different from sustaining long-context, visual, and tool-using workloads.
Bonsai 2 is most worth evaluating for local inference, single-GPU applications, and tools with cost or privacy reasons to reduce cloud calls. It should not be treated as a drop-in replacement for a coding agent merely because its weights fit. A serious evaluation should separately record the weight format, usable context length, KV-cache headroom, decode speed, and success rate on long tasks. The reported speed and quality results come from PrismML’s own tests, and the ternary assignment rule has not been disclosed. Whether the format can enter upstream llama.cpp also remains open. The practical conclusion is clear: Bonsai 2 turns “it does not fit” into “it can be tested,” but it has not yet turned “runnable” into “dependable.”