I could however totally see an economy where the chips themselves while cheap to produce cost a premium based on model and number of parameters.

Because the tech is certainly impressive and they have proof of concept. I don't know how scalable this is for them (or others), but it clearly works and shows immediate advantages. If it could integrate with existing consumer hardware, like say a PCI card you plug the chip into and switch them out when you want to change the model, anybody could easily have this at home.

But with capitalism we'd probably have to settle for DRM'd chips that self-destruct after X many tokens generated lol.

Oh man you're underselling it to the rest of the website haha.

But it's tough to understate just how fast this is without seeing it. 15,749 tokens/s is what I get, and most responses from the big models might be a bit over 1000 tokens, maybe 2000 if they're stretching it (including chain of thought). Longest I got deepseek to go recently was just a bit below 5000 tokens.

But at such speeds, all of these generation lengths - 1000, 2000, 5000 - are basically done in the blink of an eye. 5000 tokens will be written in a third of a second, or just slightly above the average reaction time.

Unfortunately Jimmy seems limited to ~1000 tokens generation so we won't be able to really push it to the limits lol.