That's not inherent, that's a consequence of performance optimizations. It's absolutely a choice to run those matrix calculations in a way that fails to have predictable execution ordering. It's just that the speed benefits to allowing that are considerable.

You can make it trivially deterministic by running single threaded on a cpu, but it's becomes too slow for practical applications if you do that.