I think you're falling victim to the just-world fallacy.