alt Hacker NewsAgreed.
To illustrate, say you live inside a fenced-off city. You say you don’t discriminate, anyone is welcome to come and talk to and trade with you. Somebody points out that there are people outside the city, behind the fence, that aren’t able to come talk to you. You are free to act or not act on that, but speaking out against the one that merely points it out and tries to change it means you take an active position to support the current discriminatory situation, rather than a passive, opportunistic one that supports whatever the political situation happens to be.
All positions are valid positions to take. They do however reflect an active choice and an active act. All of them are political. All of them come from a position of privilege, being inside the city, not outside.
No-one is "speaking out against" the one who is pointing out any wider discrimination, beyond whatever aribtrary circle you choose to draw.
What is being "spoken out against" is the idea that taking the moral (or political) action within whatever circle you feel able or willing to support is insufficient, or even discriminatory in itself. After all, this is exactly how this conversation started. Good for you if you want to change the world - let's not forget 3rd party discrimination against other 3rd parties! For many of us, it's one of numerous pressing problems to be addressed. If you wish to bring privilege into it, having the freedom to make fighting any and all discrimination a primary concern is a sign of privilege that few have.