We have computing power in our pockets a million times more powerful than we used to send man to the moon, why do you think we'll never have enough power?

Not the person you replied to, but I have thoughts on this point in particular:

1. Consumer devices have started to slow down their performance improvements because we're bumping up against the limits of physics
2. People/corporations with way more money than the average consumer will always be able to run something orders of magnitude more powerful. Any advances that improve things for the average consumer will improve things for rich people/corporations even more.
3. Training an LLM isn't really even about compute speed, it's about access to good training material. The average consumer can't afford to buy (or pirate) every book in existence like a rich person/corporation can. An average person doesn't have the ability or time to curate their own training data, but rich people/corporations do.