> Whereas in LLMs, the whole thing is undefined behavior.

Bingo!

When you exploit undefined behavior using a compiler, what do you do? You verify that its output, the object code, does what you intended with the undefined code. Or else you at least have to test the heck out of it.

Gee, what is all that similar to ...