Yep true, I'm still thinking about this as of today so my thoughts on this are not really mature and there are definitely a lot of caveats in that approach I didn't see.

However, having a rigorous spec process which ends up giving lots of details to the agent seems to me the right way to build stuff with LLMs. Rather than iterating and piling changes after changes.