There's no need for anyone in the western world to frown upon that. Literally all of us started out with the wealth that our ancestors violently extracted from colonies.
Just because you didn't personally inherit millions doesn't change the structural fact. All our industries jump-started by getting stuff from colonies. It was a big rush among the colonizers, and those who didn't do successful colonize are not among "first world" countries today.
Musk is guilty of much, but his sin in benefiting from his parents sins is different from yours only in scale.