Matrix multiplication on GPUs is non-deterministic. As are things like cumsum()

https://docs.pytorch.org/docs/2.11/generated/torch.use_deter...

This comes down to map reduce and floating point's lack of associativity. You see the same thing with OpenMP on CPUs.

People are constantly claiming determinism in LLMs that is just not there.