Middle class doesn’t necessarily mean average or median class, but rather some life style bar where you aren’t struggling even if you can’t afford many luxuries. In India, for example, the middle class is small (definitely not average!) but growing.
Having a college education could totally be an indicator for middle class even if most people didn’t have one.