TIL that 1 bit models are actually 1.58 bit with three values +1, 0 and -1
There's two variants of this (or, as the joke goes, for very big values of bit):
Ternary Bonsai 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling, giving a true 1.71 effective bits per weight.
1-bit Bonsai 27B uses binary {−1, +1} weights with the same group-wise scaling, giving 1.125 effective bits per weight.
Yeah, it's an unfortunate convention from the very first "1 bit" model. But to be clear, Bonsai comes in both ternary and actual 1-bit variants.
There's two variants of this (or, as the joke goes, for very big values of bit):
Ternary Bonsai 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling, giving a true 1.71 effective bits per weight.
1-bit Bonsai 27B uses binary {−1, +1} weights with the same group-wise scaling, giving 1.125 effective bits per weight.