So the concept is that requirements and rationale will be more permanent and important than code, because code can be regenerated very cheaply?

I think commenters here identified many of the issues we would face with it today, but thinking of a future where LLMs are indeed writing virtually all code and very fast, ideas like these are interesting. Our current tooling (version control, testing, etc.) will certainly need to adapt if this future comes to pass.