alt Hacker NewsThat doesn't seem to match the arena.ai rankings, who I'm assuming has a far larger dataset of rankings than you currently hold.
That doesn't seem to match the arena.ai rankings, who I'm assuming has a far larger dataset of rankings than you currently hold.
You're right Arena.ai has far more data. Our data is actual local coding agent head-to-head battles, so I think our quality is far higher.
Their agents leaderboard is literally just randomizing the agent and looking at metadata like tool call errors. In my opinion, it's poor quality data in high volume.
Their web dev leaderboard is in cloud containers, most people don't run agents like that.
I ran a statistical test on our data. Deepseek vs. Argon. Posterior probability DeepSeek 4.1 Flash is stronger: 99.1%.