LLMs work OK for "Mostly iterative and mostly one-off" tasks like codegen, where you can effectively "review the result into existence", and that's where most of the buzz is at the moment.

Where they don't work at all well is for hands-off repeatable tasks that have to be correct each time. If you ask a LLM for advice, it will tell you that you need to bound such tasks with deterministic input contract and a deterministic output contract, and then externally validate the output for correctness. if need to do that you can probably do the whole thing old-skool with not much more effort, especially if you use a LLM to help gen the code, as above. That's not a criticism of LLMs, it's just a consequence of the way they work.

They are also prone to the most massive brain farts even in areas like coding - I asked a LLM to look for issues in some heavily multithreaded code. Its "High priority fix" for a infrequently used slow path that checked for uniqueness under a lock before creating an object was to replace that and take out a read lock, copy the entire data structure under the lock, drop the lock, check for uniqueness outside of any lock, then take a write lock and insert the new object. Of course as soon as I told it it was a dumbass it instantly agreed, but if I'd told it to JFDI its suggestions it would have changed correct code into badly broken code.

Like anything else that's new in the IT world, a useful tool that's over-hyped as sweeping awsy everything that came before it and that's gleefully jumped on by PHBs as a reason to get rid of those annoying humans. Things will settle down eventually and it will find its place. I'm just thankful I'm in the run up (down?) to retirement ;-)