These results don’t just contradict more serious benchmarks, they are wrong on an entirely different axis. This is a saturated benchmark. Haiku gets 96%. The results here are “not even wrong” and this being #1 on HN right now is a massive smell of either bots or massive ignorance or both.

I'm seeing a worrying trend on HN. Nearly for all articles, there is one unquantified, unproven comment at the top saying it's 100% AI — no proof, just baseless emotion of what fits the commenter's writing style. This is the new witch hunt, or virtue trolling.

Look at the later comments, they have substance oriented discussions.

HN Mods - can you please consider a policy against such comments, it's overflowing the site and is diluting discourse and value. If people don't like an article, they can simply ignore it. These articles are reaching the top because enough people consider it of value.

But when an article comes to the top, and the topmost comment and discussion thread is an unqualified witch hunt, it's getting sick.

Note that in AAs report, Kimi K3 was at #1, then they updated their criteria and published a new report on the same day where it was no longer at the #1 spot. They may be under pressure not to declare a chinese model as #1.

While I was inclined to push back on the results, with Fable and Sol being so low, I have to admit I've also run into refusals several times since the latest models have arrived, and I've had to use Kimi K3 or DeepSeek to complete the task. Usually security auditing type stuff, but Fable balks at all sorts of ridiculous things, sometimes stupid things. I've even had Fable fall back to Opus and then Opus refused the task as well. So, it actually is becoming hard to use US models for everything because they refuse to work on a pretty broad selection of security and security-adjacent tasks. I guess if you're not at a Fortune 500 or a member of a fascist government, you don't get to use the best models to protect yourself and that's just how it's going to be.

But, you're right. The prose is miserable Claude-speak, difficult to wade through.

Haha theres even an em-dash in the title

Because it's notoriously hard to benchmark LLMs. Ultimately every benchmark is different and measures different things. This is why companies that make LLM models have private benchmarks - they find the areas where the model is weak and make that their goal.