In order for it to be this ubiquitous it has to run locally or on commodity hardware IMO.

LLMs as they are, can already run on smartphones, which pretty are ubiquitous themselves.

So a flagship phone would have 12-16 gigs of RAM these days I believe. A low-end phone 4 gigs.

Here are the sizes of some different parameter count versions of Qwen 3.5, a popular Chinese open-weight LLM:

27B: 17 GB - not yet possible to run on current flagship phones, but once the RAM crisis ends, I could see this happening.

9B: 6.6 GB

4B: 3.4 GB

2B: 2.7 GB

0.8B: 1 GB.

For any recently manufactured device, there will be versions of multiple popular LLMs that will run on the RAM size they have available.