XOR is the worst hashing function imaginable, especially if the data isn't truly random, because it can cancel out to become even less random!

It's better to use a KDF or something. Even a block cipher should work far better than XOR.