To the outside, the difference is hard to tell, isn't it? Between neuro-diversity and genuine unpleasantness -- isn't it mostly that one has a diagnosis (that you know of) and the other does not?

You might change your moral judgement of someone's behavior if you find out they have this or that condition (at least I do), but it doesn't change how their behavior impacts you, does it? If it did, I think the best you could do is to assume that everyone has some sort of condition that makes them act the way they do, and it'll be less of a problem.