You could run the 2-bit quantization at about 10GB for the weights. You'd probably also need to quantize context to get a decent-sized context. The Unsloth model card shows the 1-bit quantization doing seemingly shockingly well on a task (but there is notable loss in model quality for each bit you give up below about 6 bits). An MoE model can partially reside in system RAM without catastrophic performance loss, but a dense model like this one cannot. You have to run it entirely out of VRAM, or it'll slow to a crawl.

Yes. It won't be as fast as fitting the whole model into VRAM but llama.cpp defaults are pretty smart about GPU/CPU splits these days. Just YOLO it with `llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL` and it'll definitely at least run.

You may be able to run a quantized version. One strategy I've found effective is to set Claude or Codex loose on the problem and they'll do a decent job setting up the best possible model revision.