> If you ask the compiler to compile the same input, it will produce the same output.

As with LLMs, unless you ask for the output to be nondeterministic. But any compiler can be made nondeterministic if you ask for it. That's not something unique to LLMs.

> With LLMs the output depends on the phases of the moon.

If you are relying on a third-party service to run the LLM, quite possibly. Without control over the hardware, configuration, etc. then there is all kinds of fuckery that they can introduce. A third-party can make any compiler nondeterministic.

But that's not a limitation of LLMs. By design, they are deterministic.