The problem is that everyone imagines some sort of just and fair arbiter of these things, when the reality is all of the social scoring and consequences and reputations rarely actually stem from one’s actions, and far more often stem from how much money someone has put in someone’s pocket, who someone knows, the color of someone’s skin, or what’s between someone’s legs.
Until we’re actually serious about treating people equitably, (not equally, as that would simply leave the lopsided power structure we have in place) we aren’t getting out of this.
> The problem is that everyone imagines ...
I'd phrase it "most people want to imagine". Or will claim they want to - since not believing sounds depressing, or suggests that the person is an evildoer hoping to escape justice. (But unfortunately, people usually disagree about exactly what would be "just" or "fair". While wanting to imagine that they don't. Yes, the problem just went meta.)
> when the reality is ...
Try asking some really old folks about how much drama, inequity, and nastiness there often was in social settings where everyone was the same color and gender, nobody was notably wealthy, and nobody had any great connections. Or talk to an experienced junior high teacher. Or read some history. Humans are quite capable of dividing themselves into camps over any "differences" that they're able to perceive. Or invent, since dividing themselves into camps is often the unspoken objective.
I don't understand the desire to argue against this in identity politics terms, which I assume you realize is going to be extremely divisive. There's a far more simple argument that people, independent of ideological framing, would tend to agree upon. And that's what is valuable to one person isn't valuable, to the same degree, to another. For things that are completely illegal, like murder, such a system would be pretty useless because we already have systems in place to punish that sort of behavior and defacto social scoring on it as well.
So this is going to come down to things that aren't usually illegal or 'that illegal', but otherwise affect society. So for instance one group might want to reasonably punish the directors of a company that causes emissions. Another group might want to reasonable reward the directors of a company that creates a large number of desirable jobs. And even in this one example you immediately end up with a weird scenario. Is it okay to pollute as long as you make enough jobs? Who gets to decide that?
There's no need for cartoon villainy for this to be a bad idea. Even completely well intentioned, it just doesn't work out well. And the more diverse a society, the worse it's going to work.