I strongly disagree. I knew a number of people who were accepted into those elite schools (two at MIT, two at Harvard, one at Brown). All but one of them were lower-to-lower middle class and only able to attend due to the grants given by the school itself. The thing they all had in common was they were incredibly intelligent and hard working. The elitism definitely exists, but it's not the way you describe in my experience