We made an under-the-radar optimization in a data flow in my company. A given task is now much more freshData-assisted that it used to.
Was a LLM used during that optimization? Yes.
Who will correlate the sudden productivity improvement with our optimization of the data flow with the availability of a LLM to do such optimizations fast enough that no project+consultants+management is needed ?
No one.
Just like no one is evaluating the value of a hammer or a ladder when you build a house.
CONEXPO, World of Concrete, and NAHB IBS is where vendors go to show off their new ladders and the attendees totally evaluate the value of those ladders vs their competitors.
Is there a productivity improvement resulting tangible economic results coming from that optimization?
It’s easy to convince yourself that it is, and anyone can massage some internal metric enough to prove their desired outcome.
But you would see more houses, or housing build costs/bids fall.
This is where the whole "show me what you built with AI" meme comes from, and currently there's no substitute for SWEs. Maybe next year or next next year, but mostly the usage is generating boring stuff like internal tool frontends, tests, etc. That's not nothing, but because actually writing the code was at best 20% of the time cost anyway, the gains aren't huge, and won't be until AI gets into the other parts of the SDLC (or the SDLC changes).