Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
You might already know this, but a large part of test-time compute / 'overthinking' is just letting the model do more passes, and refine its activation residuals more.
For example, even if you make thinking tokens literally just '....' (absolutely meaningless; zero information), you still see significant performance improvements: https://arxiv.org/abs/2404.15758 and https://arxiv.org/abs/2607.22925 for some starters.
Treat thinking more like a "loading screen message" that's been RL'd to somewhat resemble its actual internal state; which happens in its activations, not tokens.
Did you observe the model overthinking on practical tasks? While 3.8 does think a lot on xhigh I've found that it really depends on the task. On one-shot prompts that are usually the first to be posted during new releases it will tend to spend a lot more time thinking than doing. In other words the more open ended a problem space becomes, the more Qwen will tend to second-guess itself.
Conversely I've found that it can be as succinct as Muse Glimmer when it has a clear path forward. This can be either through well defined requirements or through unambiguous steps to take based on its own reasoning. While I do think it's fair to call out how much smaller model overthinks especially on one-shot prompts, in practice it hasn't led to an overall increase in time to task completion at least for what I've been using it for.
That's low reasoning for a model, but max for a HN comment.
Since you're running through the trouble of setting that up, if its 125B params, but only 6B is activated, does that mean you mainly need to allocate enough VRAM for that much of the model? Or do you still need enough VRAM for the whole thing (and buffer for context window)? Or maybe anyone can inform me, this is one area I'm uninformed in.
You are absolutely right to push back on this. Let me think for a moment.
Yep.. for 'general purpose' use I found qwen3.8:27b to be disappointing due to overthinking. It's brutal especially considering how slow it is compared to MoE variants. It often overthinks to the magnitude of ~10x the tokens vs a ~4x faster gemma4:26b-a3b.
As a result, qwen3.8 will churn over a prompt often for 5-10 minutes while gemma4 regularly finishes the same prompt in under 20 seconds, while giving a consistent and accurate response in my favorite test case. Qwen3.8, despite churning like that, often misses with an inaccurate answer.
Obviously, 'YMMV' depending on your use case... just sharing my two cents.
What kind of machine do you have in your homelab that can run this model?!
What about adding rtk proxy?
My stack is basically deer-flow with Qwen3.5-122B-A10B; this hopefully will be a speed and intelligence improvement. Running deer-flow overnight on any research topic or verify clear scoped programming issue is really neat.
Also, heating my home during the winter is nice.
Oh, also, I use llamacpp with --reasoning-budget; very simple way to move on.
It will be interesting to see the token efficiency analysis. This is my first question now with Chinese models; I take raw benchmark performance for granted.
You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each:
1. It overthinks — Just like the previous iteration. High confidence. 2. It doesn’t overthink — Improvement from the last model for your use case. Regression for others. 3. It sometimes overthinks — Best case all around. A feature, not an impairment.
One final thing worth mentioning: (I made myself irrationally angry writing this)