Yeah, a good balance would be a local reasoning model paired w/ a knowledge pack + web-based searches. You want your obscure coding language? Provide a pre-vetted knowledge pack locally that it knows how to use. Same with any other domain-specific knowledge.
Yes especially now that local context isn't so costly anymore. With llama3 it would kill my vram just to have a few thousand tokens. With recent models it's hardly an impact anymore. Not sure how they manage that but it's great. It still takes more processing but I also run an older card that is not very good at prefill.
That’s what I do with typed, there’s a knowledgebase that loads with an embedding model for code related lookups.
For that, Qwen 3.8 Flash Next is split into two shards, one with the model, the other with the n-gram table, which functionally holds its world knowledge.
Personally I recommend ISTA-DASLab's version, pruned to half the number of experts to specialise in coding:
https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RC...
I didn't think my somewhat dated RX7700S would be supported in Strata, but it is and I'm getting 120tps+ prefill and 20tps+ generation with it, which is already practical for my purposes.