This is absolutely correct.

But I would subtly add it's also about the training data, which is optimized towards English. Not intentionally, just incidentally. (And this is a really hot area of improvement with a long way to go.)

Which is largely why the tokenization part that you mentioned is what it is.