Bonsai 2 does some additional tuning in addition to just quantizing (mostly around the fp16 block weights per 32 params), so they’re able to pack more in to just so few GB.

The tradeoff with unsloth seems to be that it’s more directly applicable for other off the shelf open models, whereas with Bonsai you’d have to wait for them to release another one

Bonsai 2 doesn't work at all, Unsloth's smallest... also doesn't work, but is still better.

Bonsai's technology would be better suited for 100s of B models.