alt Hacker NewsYou're mixing up sizes of different quants. The 60GB is unquantized, and Qwen's unquantized size is around 54GB. Their sizes as like quantization levels are similar.
You're mixing up sizes of different quants. The 60GB is unquantized, and Qwen's unquantized size is around 54GB. Their sizes as like quantization levels are similar.
From my perspective it doesn't make sense to talk about the number of parameters. What matters is model size in bytes and its performance at that certain size.
Meta actually relesed official 4 bit quants in 17GB, but I haven't seen any indication that training was quant-aware, so the quants are not going to have same performance. 3.6 27B has official FP8 quant that AFAIR was trained with quantization awareness.
The best example is last year's gpt-oss which was released prequantized in mxfp4 so 20B parameter model was under 14GB and 120B was under 70GB right away.