alt Hacker News> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM
I think there are two different claims here:
- developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims.
- LLMs give no productivity uplift at all, which is much harder to defend. A repeat of the famous METR RCT study did find evidence of improved productivity, and this seems to align with the experience of many experts I trust.
Replies
Developers are cashing in on the productivity gains. Meaning instead of using the increased productivity to do more work, they just become lazier or do fun irrelevant side projects instead, where as before there was just not much time for such things. I have definitely procrastinated on work simply because I know I can swoop in with an LLM and do it all in 15 minutes, whereas before I would have spent a few hours.
This is how LLMs can both result in greater productivity yet still not appear to yield much more benefit than the pre-LLM era.
And there is no benefit to the developer for doing the work first then just sitting idle. That’s how you get people putting more things on your plate.
> LLMs give no productivity uplift at all, which is much harder to defend
It’s not really hard to defend. Because when people says that productivity is uplifted, they are talking about amount of work, not the ROI. That’s why you keep hearing about LOC, amount of PR and prototypes, and the time taken is actually “time to PR” and not “time to production + time spent on bugs”.
Specifically my claim is "the relatively minor productivity uplift I would personally get out of agentic development is offset by the high cost, along with unresolved questions about long-term code maintainability, so I am not convinced that it is actually beneficial."
IMO the bigger problem is that ~1.5x individual dev productivity uplift seems to translate into 1.05x uplift across the team. People have been waaaaayyyyy too overconfident about this stuff.