logoalt Hacker News

peri-cl • today at 6:53 PM • 1 reply • view on HN

The big trend in that graph is that smaller models collapse the most. Everything with "-flash" in the name, plus OpenAI Luna.

Look at GPT 5.6-Sol vs. GPT 5.6-Luna in particular.

(The GLM pair shouldn't be expected follow this pattern, because they're misnamed: GLM-5.3 and GLM-5.3-flash are distinct lineages. The -flash is an early release of a new pretrain on a new architecture).

If this *is* the main factor, I'd expect Kimi K3.1 and Gemini Carbon would also do well on this test (unlike Gemini-flash); while Haiku 6 would collapse (breaking the trend of the other recent Claude models). It's a falsifiable prediction.


Replies

pianopatrick • today at 8:14 PM

That has unfortunate implications for the cost of using AI for coding.