alt Hacker NewsFrom my perspective it doesn't make sense to talk about the number of parameters. What matters is model size in bytes and its performance at that certain size.
Meta actually relesed official 4 bit quants in 17GB, but I haven't seen any indication that training was quant-aware, so the quants are not going to have same performance. 3.6 27B has official FP8 quant that AFAIR was trained with quantization awareness.
The best example is last year's gpt-oss which was released prequantized in mxfp4 so 20B parameter model was under 14GB and 120B was under 70GB right away.
> but I haven't seen any indication that training was quant-aware
readme on huggingface says they've benchmarked the quants -- for 17GB quant reported 1% avg loss across 15 benchmarks (sadly no breakdown).
I assume that's strong enough signal for QAT. Not just first party quants, but they cared to monitor degradation.