alt Hacker NewsIt will be the same as these smaller models from OAI and Ant. Strong in benchmarks but pretty bad in real world use.
I've read this same comment every time a new Luna came out, and the difference between free GPT (Luna) and paid (Sol and Astra) is still outrageous for general use.
Replies
It's definitely not the case that small models only do well in benchmarks. We're working on a battle system on top of OpenCode, so have data for real people doign real work.
Right now DeepSeek Flash 4.1 Flash has a 64% win rate against GPT 6.1 Sol. Small data still, but it has 33 wins vs. 18 losses, which is confidence interval of 52% - 78% win rate. https://agentduel.com/leaderboard/models/deepseek/deepseek-v...
3.8 flash was easily the best general everyday person model yet though
I think this is a completely different class from Luna and is much more akin to 6.1 Sol in terms of capabilities and general intelligence.