> We need to be very careful when using AI systems to evaluate people in general -- not because they might be biased (which they might be), but because even a small bias, if used by virtually everyone, can be damning.
I don't think this even requires any bias.
Assume there's some loose ordering of who is or isn't a good hire, and every employer has their own fuzzy view of it. If you get slightly better or worse as a potential hire (pick up an extra degree, let your latest certification lapse, whatever), it gets somewhat easier or harder to get hired.
Now assume that same ordering, but all employers share the same view of it. I'd expect the divide between employable and not employable to be much sharper.
Well, I'd say that specific ordering is the bias. But I see what you mean. The bias is arbitrary, but still very real.
Also, we will of course have all kinds of attempts to "game" the system to get ahead. Optimizing (even more) for the metric. Degree mills, for instance.