Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update

This is a great area of research with many players. See also ByteShape https://byteshape.com/blogs/Qwen3.8-27B/ (I'm not affiliated)

There are so many of us with weak GPUs and these updates are massive wins. Still waiting for the 2080 moment where this 27B param model and a reasonable kv cache can exist and function at slow, but overnight-burning speeds ..

How does it compare to the 3.8-27B versions from TensorFold? I.e. does it make sense to run unsloth via TensorFold engine?

Figuring out local llms , but it did built me single page html . On 12 GB VRAM and 16 GB ram windows .

Just installed Qwen 3.8 Splash tonight (doesn’t appear on linked page). Super impressed. Easily averaging 45 tps, even with thinking on. Totally acceptable. Running M4 Mac mini, 48GB, LM Studio.

How do the smallest quants of this compare to Bonsai 2?

Awesome! I wonder how it does at Q5+ compared with v3.0

[dead]