There can be also a softer version of it, which is that cultural richness and focus on education are easily transmitted within families. A society that doesn't value culture and education is going to produce less educated families with even less educated children.
It’s also true that IQ is both real and highly heritable. The military uses what’s essentially an IQ test to screen out the bottom 15% or so of the population: https://www.psychologytoday.com/us/blog/after-service/201801.... The military has found that people with aptitude test scores below the cutoff can’t be trained to competently perform any job in the military.
IQ is also highly heritable: https://pmc.ncbi.nlm.nih.gov/articles/PMC5985927.