It’s an unpleasant experience to have people who think they know you but clearly don’t project their opinions of what they think you’re like.
It’s probably a very human trait to do that but it is a bad habit.