What people "think" affects how they act behind your back. Whether they tell you all the information you need or the bare minimum. Whether you get to be picked up for a project or not in situation where there are multiple competing people. Whether you get invited and can become member of an in-group.
Basically, what people euphemism away here is "you get to be slightly discriminated against". In the USA situation, we would say "they are racist against you while keeping it politically correct wherever provable". It has measurable impact.
I know it sounds bad but as long as it is private stuff you cannot choose what people prefer whether that is discrimination or not.
In fact, if someone does not want me somewhere, I would not need to stay there to like them.
Also, how do you know it is always discrimination? Sometimes it might be but sometimes maybe no. It is a problem without a good solution that I think trying to solve brings further trouble (in these situations, I do not support any kind of legal discrimination, of course).