What baffles my mind is how the entire LLM/Transformer architecture only began in 2017 (with the "Attention Is All You Need" paper), and less than 10 years later, the entire global economy decides to dedicate all of its efforts to creating tokens for these models. The money dedicated to this is bigger than the railroad, highway, and electrification buildouts COMBINED, in both dollars and percentage of GDP. Humanity is sacrificing its energy, capital, and natural resources to feed the AI machine. In a way, it has already enslaved us. And it hasn't even been 10 years!

All the latest cards from nVidia are still GPUs. There are TPUs which exclusively do tensor calculations but most data centers are running GPUs.