logoalt Hacker News

lnenadtoday at 1:30 PM11 repliesview on HN

Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.


Replies

redrixtoday at 2:06 PM

You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each:

1. It overthinks — Just like the previous iteration. High confidence. 2. It doesn’t overthink — Improvement from the last model for your use case. Regression for others. 3. It sometimes overthinks — Best case all around. A feature, not an impairment.

One final thing worth mentioning: (I made myself irrationally angry writing this)

show 10 replies
dannywtoday at 2:50 PM

You might already know this, but a large part of test-time compute / 'overthinking' is just letting the model do more passes, and refine its activation residuals more.

For example, even if you make thinking tokens literally just '....' (absolutely meaningless; zero information), you still see significant performance improvements: https://arxiv.org/abs/2404.15758 and https://arxiv.org/abs/2607.22925 for some starters.

Treat thinking more like a "loading screen message" that's been RL'd to somewhat resemble its actual internal state; which happens in its activations, not tokens.

show 2 replies
xatnystoday at 2:53 PM

Did you observe the model overthinking on practical tasks? While 3.8 does think a lot on xhigh I've found that it really depends on the task. On one-shot prompts that are usually the first to be posted during new releases it will tend to spend a lot more time thinking than doing. In other words the more open ended a problem space becomes, the more Qwen will tend to second-guess itself.

Conversely I've found that it can be as succinct as Muse Glimmer when it has a clear path forward. This can be either through well defined requirements or through unambiguous steps to take based on its own reasoning. While I do think it's fair to call out how much smaller model overthinks especially on one-shot prompts, in practice it hasn't led to an overall increase in time to task completion at least for what I've been using it for.

show 2 replies
grim_iotoday at 1:33 PM

That's low reasoning for a model, but max for a HN comment.

giancarlostorotoday at 6:05 PM

Since you're running through the trouble of setting that up, if its 125B params, but only 6B is activated, does that mean you mainly need to allocate enough VRAM for that much of the model? Or do you still need enough VRAM for the whole thing (and buffer for context window)? Or maybe anyone can inform me, this is one area I'm uninformed in.

show 1 reply
javier2today at 4:38 PM

You are absolutely right to push back on this. Let me think for a moment.

Casteiltoday at 4:13 PM

Yep.. for 'general purpose' use I found qwen3.8:27b to be disappointing due to overthinking. It's brutal especially considering how slow it is compared to MoE variants. It often overthinks to the magnitude of ~10x the tokens vs a ~4x faster gemma4:26b-a3b.

As a result, qwen3.8 will churn over a prompt often for 5-10 minutes while gemma4 regularly finishes the same prompt in under 20 seconds, while giving a consistent and accurate response in my favorite test case. Qwen3.8, despite churning like that, often misses with an inaccurate answer.

Obviously, 'YMMV' depending on your use case... just sharing my two cents.

show 1 reply
looksjjhgtoday at 6:14 PM

What kind of machine do you have in your homelab that can run this model?!

show 2 replies
atmosxtoday at 3:32 PM

What about adding rtk proxy?

cyanydeeztoday at 1:43 PM

My stack is basically deer-flow with Qwen3.5-122B-A10B; this hopefully will be a speed and intelligence improvement. Running deer-flow overnight on any research topic or verify clear scoped programming issue is really neat.

Also, heating my home during the winter is nice.

Oh, also, I use llamacpp with --reasoning-budget; very simple way to move on.

show 1 reply
esafaktoday at 2:39 PM

It will be interesting to see the token efficiency analysis. This is my first question now with Chinese models; I take raw benchmark performance for granted.