alt Hacker NewsAs well as cost-per-task I think it's worth thinking about speed, especially in non-coding contexts that benchmark less cleanly
We've started trying to do some comparison videos to capture more of the UX vs speed vs cost stuff e.g. https://www.linkedin.com/feed/update/urn:li:activity:7479891... which one of my team did for my LinkedIn account (disclaimer: marketing)
(In this particular case Deepseek was way slower than GPT 5.5 but I think that's because it installed Libreoffice half-way through the task!)
Concrete example: I’ve been trying to use Claude to generate all my commit messages, but it takes 5-10x longer than if I just write them myself. Mine are less detailed, but one line changes are sometimes inconsequential (especially white space reformatting). I wish there was a model that understood the codebase well enough to generate commit messages in half the time.