alt Hacker NewsThey say it is trained with quantization awareness, so it should only be 15GB or so. Qwen was only trained in FP8 with QAT.
UPD, NVM, got misled by comments here. It is actually almost 60 GB so much larger
Replies
xienze • today at 1:03 PM
You're mixing up sizes of different quants. The 60GB is unquantized, and Qwen's unquantized size is around 54GB. Their sizes as like quantization levels are similar.
➕ show 1 reply
Quantization awareness doesn’t change the size of the weights, just means it won’t degrade when quantized. QAT = quantization aware training. They will both be very similar in size at the same quant.