1008
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 05 May 2026
1008 points (97.9% liked)
Technology
84646 readers
4242 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
Can someone ELI5 why they are doing this? I thought all the AI shit was in the cloud?
AI runs in the cloud because it needs a powerful server to run the biggest (i.e. "smartest") models.
The cloud servers are doing nothing special that another powerful enough computer could do, just a huge amount of data processing.
You can run an ai chat on a steam deck or directly on a phone, if it's not too demanding ("smarter" models are bigger data files, so won't fit in the memory of a small device).
Today, for instance, I had a phone call from "Spectrum Internet support" and part-way through the call my phone blared an alarm and said "possible scam" on screen.
The phone itself interpreted the conversation as sus.
https://support.google.com/phoneapp/answer/15654065?hl=en
The cloud being a bunch of computational power (servers). A bunch of phones in a network also can be utilized for said computational power. Passing the savings on to you! ;)
I recall this. Pretty much the same idea, but this time it is opt-out(or is it actually?)
They need their features to work offline too probably.
It's also cheaper, if they can offload a portion to the user's computer.
Cheaper for them, that is.
What I want to see is throttleable models, kind of like progressive JPEG, where the default model is “nano” and it has a watch function that analyzes if more tokens might be needed for a certain task and scales up as needed — identifying if the resources are too much for the device and offloading to the cloud (with explicit permission) only if (but always if) needed. Over time as the technology improves, larger models move to the endpoint.
And then people could have a basic set of sliders: on-device only, on-cloud only, or somewhere in between, based on the user’s preferences.
That's basically model routing, and has existed a while. Open AI's GPT-5 and llama-swap do that, for example. If the task is simple, it uses a smaller, less intensive model, and only uses the slower, larger one of the task is more complex.
Though most tend to operate with models on the same device/service, rather than a model run elsewhere.
In an internet browser?
Yeah, even there. A page loading is one thing, but browser features are somewhat independent of the content. There's also a good chance this is being used as a hook for other Google products like Drive or Docs (which are basically websites under the hood) to allow offline file management, creation, etc.
It's a bad choice, but it wouldn't be the first bad choice Google has made.
Well everything else is in it.
Shit, Chrome supports the use of COM ports. It's an OS within an OS.