It's more than just that. Smaller models are improving rapidly and are already "good enough" for the majority of consumer use cases. It is entirely realistic that in a few years, there will be virtually no demand for trillion-parameter LLMs. Demand for inference might not shrink in terms of functionality, but it will absolutely shrink in terms of memory and compute requirements. And then there will be a hell of a lot more server capacity available than anyone needs or wants, because they're over-investing like crazy now.

What happens when hundreds of billions of dollars spent on datacenters basically goes *poof*?

The chips already aren't being installed...

Oracle will probably go bankrupt and then get bailed out by Trump's government.

Datacenter GPUs can't game :(. As in, it's literally impossible, as they are missing ROPs.

The last generation that could is basically the Nvidia V100, and the datacenter equivalent of your Radeon VII. AMD removed the ROPs after the Radeon VII generation. The RTX 3000-generation Nvidia A100 can technically run a game if hacked to do so, but it performs extremely poorly because of a lopsided compute config.

Now, if you are interested in GPU compute and self-hosting, maybe we can use them.

But Nvidia learned to buy back and destroy GPUs during the last crypto bust, so I'm afraid they may do it again.

If I had a few billion for a TSMC factory, I would...

This is what I'm referencing. The memory sits on the package with the processor, connected with thousands of microscopic traces, so fine it needs an expensive substrate rather than a typical PCB, and expensive equipment to assemble:

The connectors are tiny, and made with immense precision. There are about a thousand reasons you can't just transfer HBM once its been used, and besides; other memory controllers wouldnt' support it.

HBM is only usable with chips co-designed for it, packaged on an interposer with microscopic traces. An enthusiast is not doing anything useful with it unless they own a few TSMC assembly plants.

The CPU modules can be tricky, too.

Now, some enthusiasts are already hacking. Go to the Level 1 Tech forums, and you’ll find a group trying to reverse engineer AMD Instinct GPUs designed for server motherboards, dealing with locked firmware, soldering stuff, you name it.

…But you probably won’t like the end goal of the hacking: using them to run LLMs and other GPGPU stuff locally. They aren’t usable for gaming.

Unfortunately, memory production capacity changes vary slowly.

In spite of the drama, memory production hasn't increased all that much, which is why there is such a RAM crisis. Maybe it will grow some before the bubble pops, but Micron, SK Hynix, and Samsung aren't stupid:

https://en.sedaily.com/finance/2026/10/01/microns-long-term-chip-deals-jump-63-percent-as-ai-memory

They are getting companies to sign long term (5+ year) pricing contracts. In other words, they clearly know the bubble could pop. They are already hedging their bets, and they aren't going to put themselves in the position of massive overexpansion.