You get a huge variability if you consider the absolute extreme outliers. Most people should be able to reach a level of competence where they can understand mathematical concepts abstractly and apply that same reasoning to other areas, and not feel a visceral rejection at the mere idea. I think that's a modest enough standard that a good portion of any given population should be able to reach, and yet education is failing at achieving that.
Your statement is not backed up by data and simply wishing it should happen isn’t a strong argument.
You probably have a narrow definition of “most people” (probably some motivated high school or undergraduate student) and too loose with what it means to “understand mathematical concepts abstractly”.
Take an analogy: imagine professional musicians saying that most people should be able to take a piece of music and understand its harmonic structure, then apply it to a new setting to generate a new piece. Most people will reject this idea as absurd.