I think stronger determinism could dramatically improve the situation here. Right now, I don't know if the same model within the same hour will produce consistent output given identical prompts and low temperature.

I have no clue what my compiler is emitting every time I hit F5. I don't need to comprehend IL or ASM because I have a ~deterministic way to produce this output from a stable representation.

Writing a codebase as natural language is definitely feasible, but how we're going about it right now is not going to support this. A vast majority of LLM coding is coming out of ad-hoc human in the loop or stochastic agent swarms. If we want to avoid the comprehension gap we need something closer to a compiler & linker that operates over a bucket of version-controlled natural language documents.