If you fix the random seed you use for sampling, your LLM is perfectly deterministic.

And there's no requirement for C compilers' UB to be deterministic either.