You bring up some good points. Just FYI my research was specifically on ai, but a very specific branch in college. So not llm but only somewhat llm flavored.

The are already including ai chips in consumer hardware. And your thinking of llm specific limitations. But the smaller models can absolutly be thrown into hardware. Its just the algorithms are moving so fast that the hardware needs to be flexible enough. Thars the biggest reason we dont see more hardware faster than gpus. Hooe that makes sense!