logoalt Hacker News

seizethecheese • today at 7:09 PM • 3 replies • view on HN

I was skeptical Sol 6.1 actually over-engineers compared to Opus 5.5, as I just switched from one to the other and haven't noticed a difference, but it looks like you're probably right!

We're building a battle system for OpenCode (https://agentduel.com/leaderboard) and have data on how large the diff was compared to opponents for both:

Opus 5.5: 0.42x (based on 299 battles) Sol 6.1: 1.42x (based on 265 battles)

Data on this is still pretty small. P=0.14. It's also affected by large differences, this is average not median.


Replies

yegle • today at 8:16 PM

FWIW Carbon is listed in the leaderboard but not in the GitHub repo.

➕ show 1 reply
jeffybefffy519 • today at 8:00 PM

This is pretty clever, i like it.

seizethecheese • today at 7:24 PM

Oh man, Sol goes nuts on tests. I just checked the test wriging data. Sol 6.1 writes 3.14x more tests than it's opponents.

➕ show 2 replies