If someone released a 0.1bit quant I can run it. /s
Fwiw, the Qwen 3.8 27B at 4 bit quant is apparently quite good and is probably the best recent model. It runs (iirc) on 18-22ish GB of memory.
alt Hacker News
Fwiw, the Qwen 3.8 27B at 4 bit quant is apparently quite good and is probably the best recent model. It runs (iirc) on 18-22ish GB of memory.