logoalt Hacker News

Diogenesiantoday at 1:04 PM3 repliesview on HN

I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM productivity gains, which still seems true for agentic AI: https://metr.org/blog/2026-05-11-ai-usage-survey/ It is particularly striking how similar the results are to LLMs before agents.

Along with the total absence of long-term data, I think the benefit can be (weakly) denied. Maybe not in the employmemt marketplace, but certainly for myself.


Replies

qsorttoday at 1:14 PM

> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM

I think there are two different claims here:

- developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims.

- LLMs give no productivity uplift at all, which is much harder to defend. A repeat of the famous METR RCT study did find evidence of improved productivity, and this seems to align with the experience of many experts I trust.

show 3 replies
deatontoday at 2:38 PM

I think the real disaster is that once you let the LLM work on a project for a bit, you start to lose understanding of what exactly is even happening under the hood in the project. You can take steps to mitigate this, but agents don't exactly encourage the behavior required to maintain a good understanding of what's going on.

show 1 reply
pydrytoday at 1:10 PM

The productivity depends upon the requirements.

If slop is fine (and sometimes it is), the benefits are undeniable. If the dev was the kind that would have produced slop anyway - again, undeniable boost.

If the quality needs to be high I think it actually can slow you down, though.

show 1 reply