You can, but that's not the kind of LLM the meme is talking about. It's about the big LLMs hosted by large companies.

Well that's sort of half right. Yes you can run the smaller models locally, but usually it's the bigger models that we want to use. It would also be very slow on a typical gaming computer and even a high end gaming computer. To make it go faster not only is the hardware used in datacenters more optimised for the task, it's also a lot faster. This is both a speed increase per unit as well as more units being used than you would normally find in a gaming PC.

Now these things aren't magic, the basic technology is the same, so where does the speed come from? The answer is raw power, these things run insane amounts of power through them, with specialised cooling systems to keep them cool. This comes at the cost of efficiency.

So whilst running a model is much cheaper compared to training a model, it is far from free. And whilst you can run a smaller model on your home PC, it isn't directly comparable to how it's used in the datacenter. So the use of AI is still very power hungry, even when not counting the training.

True, and that's how everyone who is able should use AI, but OpenAI's models are in the trillion parameter range. That's 2-3 orders of magnitude more than what you can reasonably run yourself

Yeh but those local models are usually pretty underpowered compared to the ones that run via online services, and are still more demanding than any game.

I compared the TDP of an average high-end graphics card with the GPUs required to run big LLMs. Do you disagree?