alt Hacker NewsEvery single time I click on these announcements the frontier model ALWAYS scores around ~60-80%. Every. Single. Time. Like, I just went back to the Opus 4.6 + 5 announcements, they were also scoring 60-80%.
https://www.anthropic.com/news/claude-opus-4-6 https://www.anthropic.com/news/claude-opus-5 https://openai.com/index/introducing-gpt-5-3-codex/
For every single announcement, not only do the top models always score 60-80%, the older models somehow start scoring 20-30% (even though previously they were scoring 60%). It's completely unclear whether the methodology changes, how these scores are even evaluated or which domains are being tested. What's worse is that the "benchmarks" keep being updated with no explanation or due process.
Replies
The benchmarks have a lot of churn because the models keep saturating them. But the Anthropic announcements you linked, and the current one, have the benchmark versions listed right there in the table.
A real shell game. They keep changing benchmarks
There is plenty of explanation, if you actually go seek out the benchmark. Past 80 benchmarks start approaching saturation and are not very useful to show differences