I don't think the average person hate autistic or nerdy people.
Yes, they do. They wouldn't say it, but look at how autistic people get treated by their peers, teachers, and bosses.
Obviously everything is a spectrum but I agree. If anything, "a touch of the 'tism" is the new "I have OCD" because you like a clean desk, something people are fine with saying without regards to it being true.
They absolutely don't.
The average person is probably uncomfortable around autistic people because they don't know how to deal with them and, when you see someone like that, it's usually best to avoid interacting with them. Not because autistic people are dangerous but people acting out of the ordinary sometimes are.
The lumping in of nerdy with autistic is ridiculous too. The average person just doesn't care about your interests unless they are in common. Nerdy typically just means having a niche interest or hobby.
Your parent reeks of a persecution complex.