alt Hacker NewsI have read that "pro"/"opus"/etc models can actually be worse for everyday coding as they reason "too deeply" and turn over too many stones over-thinking the problem and potentially getting distracted.
This feels absurd to me (my gut is "I want the SMARTEST model I can get!!"), but often I find that my experience of using a flash/sonnet model for every-day workhorse coding they are better.
Its not the same thing, but when I think of that I am reminded of working with some engineers in the past who are incredibly smart and have PhDs (or to put it another way, over-qualified) and they were crap engineers because they'd just not be able to focus on the task and ONLY the task at hand and would get easily distracted by the "why" or "more interesting" things when I just asked them to fix a simple bug or whatever. Again, its not the same thing at all, but it certainly comes to mind when I think of this or experience a pro/opus model suggesting we make huge refactors when a tactical fix is all that is required etc.
Of course, the opus-sized models are great when it comes to huge comprehension/research/debugging efforts where the deeper reasoning is actually useful.
Replies
That does not match my own experience, which is why I wonder if Google has evidence of that.
Consistently, lower intelligence models provide worse results in my own work. But I don't have evals on my side, just vibes.
This is my experience at least.
True, if you have a codebase that works in practice but has dozens of loose ends and poorly defined edge cases than it can chase off into rabbit holes because "oh wait, what if x is undefined instead of null? How is y defined? This outdated package has long known severe security holes and should not be used anymore, do we actually need it?".
I find Fable completely unbeatable for anything code-related. It's the only frontier model that seems to come with sane defaults.
If it implements something simple like a file export, it just knows that the file should have a meaningful name. Vibecoded feature beats most software's lazy "untitled.png".
So, yes, I want the smartest model even for simple stuff. Maybe especially for simple stuff because the tokens burned will be trivial so the cost doesn't give lower models a comparative advantage.