I have 3+ years of experience working on projects that integrate LLMs into the "old" approach to building software. Overall, whenever clients used LLM to analyse inputs without expecting them to be accurate, we were getting decent results. Not so much on the generative side, where LLMs are still producing crap output. This is on projects where humans wrote most of the code.

Things look way worse when working with vibed code. Over the last twelve months or so I encountered a number of issues with code and tests given to me by people with little or no coding experience claiming that this is a finished product and expecting me to "productionise it" (their words, not mine). Because they believe in LLMs like religious zealots and have no idea what programmers with actual experience of writing code are talking about there is a breakdown of communication and a lot of misunderstanding. On my latest project I was given a zip archive and asked to "make it work" by someone who calls themselves a "software architect" but has no idea how software development works. That person was surprised when we asked for a git repository, requirements, and design. None were given and we were told to use Claude to explain what the code is supposed to do. When we pressed for requirements the "architect" sent us a 4-page list of requirements based on the LLMs analysis of the contents of the zip file. It was so vague that it could apply to any project. Similarly, the architect vibed tickets in Jira and we had to clean them up as well, because of their vaugness and repetition or generic requirements. The three of us spent two months turning this turd into a piece of software with working tests and working implementation before we were told a PM will be assigned to our team and we will have to explain to him how the software works and what the requirements are. I am no longer on that project. I have had enough.

Another client forced addition on an AI PR reviewer. After initial excitement every dev on a 200+ team started ignoring it, because the fixes it suggests actually break code.

The marketing around LLMs convinced idiots that they are capable of producing software and that the greatest enemy of delivery are actual software developers with experience of delivering working software. The lowest point must be the recent series of ads I keep seeing on YouTube in which one character gets overly excited because he vibed a to do app... Every OS comes with one and if you don't like them, there are plenty of free and paid alternatives that cost less and work better than "your" vibed app.

I can't wait for the VC money to run out and the LLM madness to go to hell.