There are models that don't need to consume the entire worlds data infinitively. We've had then before llms, things like the pi ai camera can do ai processing on image with something like 8mb of on board storage.

The problem is anthropic, open ai, who ever are trying to do the stupidist approach and assume they can brute force money and power at the problem and don't understand that they need exponentially infinite data in a way that is unobtainable.

In an ideal world there would be a million little small local models that so specific tasks and can be ran on consumer hardware.

Qwen3.8-flash-next broke the vram requirement with its new architecture (which is a preview of their 4.0 architecture). You can run a 177b parameter model on 12gb VRAM, 64gb ram and 80gb SSD at 30tok/s and the speed scales almost linearly with increases to vram.

Incidentally, this was proven at almost the exact time all the calls for "regulating local models because they'll spread into the toasters" started.

You absolutely can have local models that run on consumer grade hardware, and the proprietary companies know it and are scared to death.

maybe they can do like 60% of the big ones?