Compared the the majority of the population in the US, and like 95% of the population in South Africa, he seemed to grow up pretty damn wealthy. Musk himself told stories about his father having stacks of cash.

No, that isn't what made him rich now, holding onto paypal shares did, but part of the reason he could hold onto those shares is because he had a wealthy family supporting him so he could just sit on them.

If you don't think Musk grew up wealthy after listening to what he says about his childhood, I could only assume you also grew up in an extremely privileged environment and just don't realize how far above the median your life was.