True but let's think about whether this actually matters in practice going forward, given that today, Deepseek/any OS-27B variant can do all the things that most devs need it to.

Let's scale this out 3 years into the future: - Opus 6.7 costs an arm and a leg to run for 2 prompts - Astra 7.89 literally requires you to buy-now-pay-later because it's so expensive - As a bootstrapped dev, your only option is serving local Qwen-4.04-27B that runs on your old RTX (that you bought in 2024)

Sure, it's worse performing. But if you can't afford the best ones in the first place, what's so bad about having a "pretty decent" model that you run essentially free (only paying electricity) if it does what you need it to?

... and if we're all in the permanent underclass anyway, what does it matter that your local model is dumber than the SOTAs? At least you have one! Some people who can't afford GPUs still have to type emails by hand.

(jk, this is actually a cool dataset though ^)