That certain countries both now and in the past have had significantly higher mathematical ability among the general population and much higher proportions going on to further study suggests that ability isn’t innate but that people don’t choose it. In the Soviet Union more time was spent teaching mathematics and a whole culture developed around mathematics being fun.
Why would ability not be innate just because some people with the ability don't use it?
Or more specifically, two of my friends teach special needs children in the 50 to 70 IQ band. Who are we going to blame for them not becoming mathematicians? The teachers, for not unlocking their hidden potential? The kids, for not trying hard enough? Claiming that the only thing holding them back is choice seems as cruel as it is wrong, to me.
Yeah, we're probably not cultivating anywhere near the potential that we could, but I personally guarantee you I am not Ramanujan or Terence Tao.