This is an excellent point! People often forget that something uncommon out of a much larger pool is still larger than anything that comes from a smaller pool (base rate neglect).

https://www.simplypsychology.org/base-rate-fallacy.html

> For example, given a choice of the two categories, people might categorize a woman as a politician rather than a banker if they heard that she enjoyed social activism at school—even if they knew that she was drawn from a population consisting of 90% bankers and 10% politicians (APA).

The general population is much larger than the population of child prodigies.