alt Hacker NewsQwen3.8-Flash-Next
Comments
Didn't expect it to beat 3.8 27B so cleanly.
Opus 4.6 Max self-hosted at 30 tok/s on a 5k Macbook in Aug 2026. The LLM timelines are crazy.
For the impatient, I merged llama.cpp tentative branches to get it running here https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev..., thing runs at 23.54 token/sec and my setup runs at high 30 the 3.8 dense 27B.
and this is the pelican from the iq4_xs model https://github.com/alainnothere/llama.cpp/blob/disk-cache-ev...
In initial testing on Ryzen 395 / Strix Halo it's about 22 tokens/s generation and the output quality is impressive. Better and faster than 3.8 27B, and enough better to justify moving away from 3.6 35B even though 35B is still faster. The Unsloth weights don't come with vision support but you can add that back yourself. llama.cpp recipe in case anyone else wants to save some time on setup: https://pastebin.com/fcqsbDTv
> Qwen3.8-Flash-Next features a 125B-parameter main model, supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token.
Didn’t see this mentioned yet. I wonder what this means for the effective size. It’s evidently ~176B paramètres, but how does that get quantized. A 4-bit quant under 100GB seems unlikely, I’m suspecting this won’t run in 128GB unified memory
In principle I like the idea of trading more memory for compute though, even if there’s a memory shortage right now
I ran some pelicans at the four different reasoning levels (none, low, medium, xhigh - apparently high and xhigh are aliases of each other) on a DGX Spark using Unsloth's unsloth/Qwen3.8-Flash-Next-GGUF (UD-IQ1_S):
https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
Surprised I didn't get one I liked as much as the Qwen 3.8 27B one https://simonwillison.net/2026/Aug/16/qwen-38-27b/#the-defau... , maybe because of quantization.
I can't imagine the future any more. US companies playing it safe and control models releases. Chinese companies are just like open source everything.
It's like Chinese are incentivized to open source from day one (years ago). While most US companies are deciding in realtime.
It's crazy that we need both to survive and advance further in the future we have never imagined.
Waiting for llama.cpp support to land, but this might be a big deal for Strix Halo users.
6B active params helps around the memory bandwidth constraints, but a 128GB box can probably run the Q3/Q4 quants fairly easily with a decent context size. This might actually be better for strix users than 27B, which was already very good.
this is a new architecture (foreshadowing qwen 4)
> trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board
Can someone explain the intuition behind the en-gram idea? I know DeepSeek published a paper about it a few months ago and the Gemma models have a lightweight version of it; but it hasn’t clicked for me yet
It's in Unsloth Desktop already. Looks like it's 73GB, so 128GB Mac or Strix Halo etc will work. Exciting!
If you have a DGX Spark, try my Spark/SM12x specific inference engine.
I've got it (Qwen 3.8 flash next) working (sans ... MTP working on that now).
https://github.com/rdaum/eider/
~80tok/sec prefill, 12tok/sec decode, ~80GiB memory resident, the n-gram table pages from SSD.
Those benchmarks look seriously impressive.. considering how small of a MoE model this is.
How is input token efficiency/verbosity on this model? Has anyone tried? GLM 5.2 was doing lot of turns and thinking piling up input tokens in the context (compared to Claude and GPT models). Then Qwen3.8-27B was 2x of that. Both delivered good output results but those cumulative input token costs were not cheap. Note this is on our specific business workloads. Genuinely interested in other people's experience (if you are able to try it out).
Interestingly they also share the parameter count for Qwen 3.7 Plus (397 b a17b). I don't think these were known before but I might be wrong.
FYI: nothing seems to be able to run this (easily) yet. llama.cpp, vllm etc I couldn't get working because of no support in the mainline version.
Does anyone have an idea how this might perform on a DGX Spark at longer contexts? I've been trying to investigate their performance with these medium-sized MoE models, but I'm seeing a lot of incomplete and conflicting information. The 273 GB/s bandwidth looks awfully bad on paper...
It looks like this also undercuts the already absurdly inexpensive Deepseek Flash in pricing. Wild.
Unsloth doesn't have all quant versions yet :-(
looks like it's better than deepseek v4 flash
Also announced GLM-5.3-Flash: https://news.ycombinator.com/item?id=49449507
Father, I cannot scroll the website.
very interesting. new architectures is the most interesting type of news. after what i experienced when gpt-oss came out i have been on the look out for architectural approaches that improves efficiency.
Pelican: https://gist.github.com/SerJaimeLannister/8fdef9c00175da0ca6...
Aside from the pelican, I am sort of impressed by the fact that things are going the way in terms of really impressive small models.
Also I love how this uses N-gram embedding. I think that Longcat was the first one who used it (I submitted that submission on hackernews because I really just loved the idea of it that I understood), I am certainly more interested in local LLM models and its interesting how they are utilizing new architectures to do some really impressive optimizations!
(Do note that I created it using a free rate limited end-point that I found on the huggingface space section: https://victor-chat-with-qwen3-8-flash-next.hf.space)
Definitely need to try this out locally.
Will this be cheaper than DS4flash ?
I think these "Flash" models are sort of an evolutionary dead end. Sure, there are some routine tasks and applications where they can be used. But for the actual novel development work? It's much better to run a big model at high power for 30 mins than watch the Flash model struggle for 2 hours and produce massive churn.
Same reason your phone has a few big CPU cores for real work, it's much better to "race to idle" than have an "efficient" core struggle. Shitty experience, shitty power efficiency.
what's the deal with absent scrollbar on the website?
Aaaaaaaaaaaaaand dario meltdown on twitter in 3, 2, 1...
Yoh yoh I like It
[dead]
do we really need breaking news about qwen posted every single day?
Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.