alt Hacker NewsFwiw, the Qwen 3.8 27B at 4 bit quant is apparently quite good and is probably the best recent model. It runs (iirc) on 18-22ish GB of memory.
Fwiw, the Qwen 3.8 27B at 4 bit quant is apparently quite good and is probably the best recent model. It runs (iirc) on 18-22ish GB of memory.
quantization has come a really long way. Q3 is fine, especially GSQ-RCO. You can run 27B well on 16GB of VRAM.