logoalt Hacker News

UncleOxidanttoday at 4:25 PM5 repliesview on HN

That's a 2-bit quant of DS4 flash. You're probably better off running Qwen3.6-27B at Q8.


Replies

spmurrayzzztoday at 4:35 PM

I think its good advice to test both on your own evals for sure, but the MoE parameters are already natively FP4 in ds4. Dropping to 2bpw isn't as big of a loss as it seems (and as corroborated by antirez's work).

Its also only 13B active, so your decode speed would be nearly 2x that of Qwen3.6-27B. So there are other latent benefits as well.

show 1 reply
Catloafdevtoday at 4:50 PM

For most coding or agentic tasks, Qwen 3.6 27B likely outperforms, yes.

For 'general intelligence', DS4 Flash seems to be a noticeable step up still.

sosodevtoday at 4:30 PM

I suspect it would depend on the task. DS4-flash does, as previously mentioned, handle quantization very well. Even at 2-bit it's still very coherent.

FuckButtonstoday at 5:48 PM

qwen 27b at q8 is slower and worse than ds4 at q2 in my experience.

ckocagiltoday at 5:02 PM

Isn't Q8 way overkill these days? I see many graphs showing Q4 or Q5 having less than %1 deviation. Nvidia's NVFP4 Qwen quantization should be even better due to its better training methods.

show 2 replies