The last line of GP's comment is key here: "Who do I sue if Palantir decides I am an illegal?"

This shouldn't make as much of a difference as it does, but due to how our legal system works, it's much harder to get meaningful legal satisfaction when an algorithm (or other inhuman distributed system) commits a crime against a person than when a person does so.

"Computer says no", look it up.

Cars measure success by not hitting things.

Cops measure success by number of people they arrest. Note, not the number of people found guilty, that's the prosecutor.

Cops will gladly use a hallucinating computer system to beat the absolute fuck out of you with qualified immunity.