logoalt Hacker News

seizethecheese • today at 6:50 PM • 0 replies • view on HN

Nice incremental benchmark improvement at the same cost GPT‑6.1 Sol -> GPT‑6.2 Sol:

FrontierCode 2.0 39.0% -> 43.8% vs Opus 45.5%

Terminal-Bench-Science 0.1 57.0% -> 68.0% vs Opus 63.3%

Artificial Analysis Intelligence Index 54.6 -> 57.3 vs Opus 59.1%

Sol 6.1 when it was released showed really nice benchmark improvements [1] but my impression from HN has been that people haven't found it to be a very nice model [2][3].

To try to close the gap between benchmarks and real world SWE use, we've been building a battle system on top of opencode [4], and Sol 6.1 has been a real laggard there. Sol 6.1 is statistically equal with DeepSeek 4.1 Flash at 10x the cost and 25% slower.

[1]: https://artificialanalysis.ai/articles/gpt-6-1-sol-replaces-...

[2]: https://news.ycombinator.com/item?id=50129992

[3]: https://news.ycombinator.com/item?id=49897054

[4]: https://agentduel.com/leaderboard