Seems like the same idea as treating your LLM as a compiler.

When you write code in Rust and user your compiler to produce an x86 binary, you don't maintain the binary. If you want to make a change, you toss away the old binary, change your Rust code, and recompile.