logoalt Hacker News

ein0p • 01/21/2025 • 0 replies • view on HN

My models are both 4 bit. But yeah, that could be - small models are much worse at tolerating quantization. That's why people use LoRA to recover the accuracy somewhat even if they don't need domain adaptation.