Qwen3.8 27B scores 52 on Artificial Analysis

As a comparison, Qwen3.6 27B scores 38, which was the highest in its small model category (4B–40B).

Qwen3.8 27B beats all medium models (40B–150B). It has the same score as DeepSeek V4 Flash 0731, which ranks #5 in large model category (> 150B).

Sources:

- https://artificialanalysis.ai/models/open-source/small

- https://artificialanalysis.ai/models/open-source/medium

- https://artificialanalysis.ai/models/open-source/large

And once again, Qwen 3.8 27B beats Opus 4.6, what the hell.

It's both funny and a bit terrifying and I still can't quite believe it. It runs decently on a gaming PC! Opus 4.6 came out only 6 months ago and was then broadly considered the new SOTA by a comfortable margin! How in hell did they package capability in the ballpark of a Feb 2026 frontier SOTA into 27B?!

More importantly, what's the point of building monster-scale data centers on unprecedented amounts of debt when a more than good enough model runs on a GPU from a couple years ago?

The coming months are going to be exciting, that's for sure...

I used this a lot over the weekend, and it's a really intelligent and strange model.

It gets really agentic at the higher reasoning levels. It does the basics like goal tracking and tool calling well, but more than that, it gets obsessed with solving problems and will do insane/unusual things to get to the solution. It actually reminds me of GPT-5.6-Sol-max which is similarly obsessive.

It doesn't surprise me at all that it outscores Opus 4.6. Opus had way better world knowledge but was more "human" with agent stuff - sort of lazy and uncreative, basically giving up once the obvious solutions failed. These newer models work more like magic, they are so creative and persistent at finding ways to get to the solution.

I used Qwen 3.6 27B extensively (>1B tokens) and DeepSeek V4 Flash (the older one also 2B+ tokens).

And I just can't fathom that the new 3.8 beats the new DeepSeek V4 Flash (which, in my eyes, is one of the best everyday coding models).

What an insane release, and convenient size to use every day/locally.

but i will test this model extensivly.

I have an internal automated benchmark, which roughly follows my workflow, and I've been testing various models on it, local and cloud. Qwen 3.8 27B did awesome. Its understanding is correct, research is better than e.g. glm's (and I like glm), and implementation is good and careful.

Qwen 3.8 27B doesn't look benchmaxxed. These "52 AA score" numbers feel real, which is surprising. I've been using it locally for a few days for other tasks as well. If not the speed, I'd be totally happy to use it as a daily driver instead of cloud models, it is that good.

--- (benchmark, to get an idea):

1. First, initial prompt which is not super precise - similar to how I'd write a task when talking e.g. to Opus. I'm describing an idea, and asking model to come up with some plan, and also to criticize the approach. Task is about implementing a particular pi extension. I'm checking if a model actually understands what I'm asking.

2. Then, as a follow-up, I ask to research alternative implementations, research UX of similar extensions, etc. It needs to do web searches, inspect open source codebases, read articles and papers, etc. I don't prompt to do this exactly, but I expect good models to figure out they need to do it.

3. Then, implementation.

Also, one finding: Q4 and Q8 seem to have very different behavior in this benchmark. Q4 produces 2-3x thinking in the end, and makes more turns - it seems it makes more mistakes, and needs effort to recover from them, while Q8 gets more things right in a first try. In the end, quality is roughly similar, but Q8 gets there much faster, especially the implementation (tried it several times). Could be a difference between concrete artifacts, or between runtimes, I don't know, but be careful - it seems the real-world experience with qwen 3.8 27B can be vastly different, depending on how it's set up.

Regarding DeepSeek 0731 vs Qwen 3.8 27B. On this benchmark, Qwen understand my intent better, it's better at research, and I also liked its implementation more. But: if you're more precise in what you ask, 0731 is also very good, and it's quite a lot faster on mac; raw speed is better, and it needs less thinking to get there. So, I'd say it's a tie in practice, both are awesome :)

The smaller these frontier-nearing models get, the more I'm reminded of https://en.wikipedia.org/wiki/Lottery_ticket_hypothesis

Why is it so small, but expensive?

Open Router

Input /M $0.45

Output /M $3.20

Cache read /M $0.05

Throughput 27 tps

It would be a very nice model at 200-300 tps and if it was dirt cheap. What's the limiting factor of optimizing speed and price for inference providers?

For more context, this puts it on par with models like GLM 5.2 and GPT 5.6 Luna, which are far larger

China cleaned house these past few months. Kudos to them.

I would really like to see some open source US companies out there.

Perhaps model size and reasoning length trade off to some extent, similar to CPU vs. RAM. A smaller model with a longer reasoning trace has more intermediate structure to latch onto and build on.

It's 7th (!!!) overall on the agentic index, above Terra.

Why are hosting providers charging to much to host it, compared to much larger models? https://openrouter.ai/compare/qwen/qwen3.8-27b/deepseek/deep...

I want to highlight its (1-hallucation rate) at 70%. BRAVO! For me, this is its most wonderful score. GPT-5.6-Sol sits at 8%.

Why is it not included in the Pareto line intelligence/cost chart?

I'm impressed with the score. This is a model that runs on a good, but still regular, desktop PC.

Unbelievable. Bravo Qwen team.

Interesting to see where local models are going to be in the coming days. I am already starting to believe open source models are the way to go in the coming days. With Qwen 3.8 Max, Kimi K3 etx already delivering at part perf with frontier models, the future is going to be exciting.

A lot of the benchmarks seem often near meaningless these days - really bench-maxxed to the hilt. I tend to still look at the Artificial Analysis rankings to get at least an idea on relative performance of models, is that still warranted?

What or other opinions on how representative the AA rankings are of real-world performance? Any better indicators?

I can't shake this the existential feeling that this compact series of 27G bytes represent something profound and universal.

Very interesting. I was not expecting anything close to this.

Anyone try the 9B/2B distills yet? Wondering how they do for local tools

So it's effectively on-par with GLM 5.2 and GPT 5.6 Luna?

Where's GLM 5.3 score?

It tied with Luna/max. Simply incredible.

Imagine this, and sucesor models on Cerebras or other silicon...

[flagged]

[flagged]

[flagged]

Monster pure platinum cable for $100,000 per yard scored 3.1415927 on the audiophile benchmark!