alt Hacker NewsIt's great to see another competitive open weight model. Congratulations!
But as is, it's probably hard to become popular amongst hobbyists and local model enthusiasts like Qwen's 27B class models or MOE models because it's a 65B dense model and inference will be prohibitively slow on consumer grade hardware, and seems like the goal/mission of the company behind this model is to automate chip design.
So if I have to assume and connect the dots, this is for the companies who want to use AI in air-gapped environments for chip design so that they protect their IPs (not leaking to third party AI labs/hostile parties) and can afford lots of server grade inference hardware for a quick unlock with the big dense model.
Neat!
Replies
Maybe is a bit more niche, but I'm pretty sure there is plenty of people who have enough RAM to run dense models like this one, I don't know how slow is slow but people are willing to sacrifice speed if it means they get a good model running locally.
Look at "SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF" on HuggingFace, It says it has 3.5M downloads this month and its offering a model that is 2-bit and 3-bit quantized (66GB - 83.9GB).
If this gets quantized to 4-bit (Q4_0) which usually there is very little loss in intelligence compared to the base model, then the model will probably be around 40GB, and looking that people are downloading a model that is at least 20GB bigger, then it might allow even more people to run a bigger model.
But it will only be popular among hobbyist if its as good as it claims to be.
Is it competitive overall or just competitive when compared to "U.S. Open-Weight models that are otherwise pretty decent at agentic coding"? The latter does not imply the former.
What'll be interesting is how it performs relative to Qwen3.8 27b when quantised to the same size. Which, of course, is a comparison they don't want to do.
> it's a 65B dense model and inference will be prohibitively slow on consumer grade hardware
Times are changing and with MTP and DFLASH, dense models are back in style. For qwen 3.8 27b, you can get great token generation rates with MTP and for quality, it's much better than a MoE of comparable size.