alt Hacker NewsThis looks great! I also think speed should be part of the metric (i.e. how long does the model take to actually solve a task). For me, I prefer to run expensive models such as Sol on light reasoning, which usually gives me good answers with quick responses.
For my style of coding (quick back-and-forths and corrections) it makes a big difference if a model comes back in 1-2 minutes compared to 5-10, and I am happy to pay a bit extra for that.
I agree on both: indeed a nice article, and I'd like to see a chart with speed as x axis.
As someone who only needs AI for a couple of tasks per day, I don't really care how much it costs, especially when subscriptions are subsidized. I want to filter by speed (eg, max task time < 1 min) and then choose the intelligence I need for the task. This will surface models like Gemma4 31B (xhigh) running on Cerebras and GPT Sol (med) fast mode. Using these models feels great and are affordable for infrequent tasks.