I think you're arguing a different thing than determinism.

If I ask an llm to "add 2 and 2" is and it replies corectly, then I ask for "the sum of 2 and 2" and it replies "banana" that is a lack of predictability and consistency but not a lack of determinism.

As long as it produces the same output for a given input, unhinged or not, it is deterministic.

Your example at the end of different systems feeding data to each other is non-deterministic only at the system level, not the individual llm level.