Agreed. Some strategies that seem to help exist, though. Write extensive tests before writing the code. They serve as guidance. Commit tests separately from library code, so you can tell the AI didn't change the test. Specify the task with copious examples. Explain why yo so things, not just what to do.
Interesting, I haven't tried tests outside of the code base the LLM is working on.
I could see other elements of isolation being useful, but this kind of feels like a lot of extra work and complexity which is part of the issue...
Also: detailed planning phase, cross-LLM reviews via subagents, tests, functional QA etc. There at more (and complimentary) ways to ensure the code does what it should then to comb through ever line.
Yeah, this is where I start side-eying people who love vibe coding. Writing lots of tests and documentation and fixing someone else's (read: the LLM's) bad code? That's literally the worst parts of the job.