"Prosperity itself lowers fertility"
This is not true. Women entering the workforce instead of having babies earlier in life lowers prosperity. In our society women working during those early years creates more prosperity (two incomes) but those who are very rich like Musk has no issue producing a big stable of kids.
I don't believe that there is a single case in world history where increased family income did NOT reduce the number of children per family. Likewise with improvements in child mortality.