100% agree that the more deterministic code the better up to the limit where you need the LLM's ability to be non-deterministic to kick in.
There is this ACM blog post called "Manual Work is a Bug" [0] that was originally written to help humans automate processes using code. I find it just as applicable today as when it was written. You and the LLM look at what has to be done and then figure out the scripts/tools to make it happen. You then tie those tools into a system.
The more I use the above the more it makes sense and the worse the whole "just commit the prompt" seems like nonsense.
This is an interesting share, thanks. Yes, that is my mental model. Use coding agents to generate more "programs" (scripts) to automate everything. Have edge case handlers - and these handlers can develop/update the original scripts.