logoalt Hacker News

woadwarrior01 • today at 7:06 PM • 3 replies • view on HN

From the model card:

> The model is distributed as a single checkpoint of about 20GB, in mixed NVFP4 and MXFP8 precisions.

What's the point in quantizing a QAT model? Just for social media karma?


Replies

gavmor • today at 7:19 PM

Just to reiterate the question: QAT approximates quantization-errors in full-size fine-tuning, so that the model itself is insensitive to (or accommodating of, I'm not sure) them when quantized to the corresponding format.

Quantization to a non-corresponding format promises none of those accommodations.

asaddhamani • today at 7:46 PM

QAT models are trained to quantize well? I don’t get the concern here.

➕ show 1 reply
Prospernwo • today at 7:45 PM

[flagged]