30
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 06 Jul 2026
30 points (96.9% liked)
chat
8632 readers
178 users here now
Chat is a text only community for casual conversation, please keep shitposting to the absolute minimum. This is intended to be a separate space from c/chapotraphouse or the daily megathread. Chat does this by being a long-form community where topics will remain from day to day unlike the megathread, and it is distinct from c/chapotraphouse in that we ask you to engage in this community in a genuine way. Please keep shitposting, bits, and irony to a minimum.
As with all communities posts need to abide by the code of conduct, additionally moderators will remove any posts or comments deemed to be inappropriate.
Thank you and happy chatting!
founded 5 years ago
MODERATORS
Yes it is a bubble and will pop. Yes it can be done sustainably. I literally have a local LLM made by Alibaba (Qwen) that runs on my laptop NPU and is still pretty useful for Linux debugging. Now it's made to be the like normal chatbot shit which is dumb, but it works. If you took a similar sized model and trained it specifically to be a helper for a specific distro (like train it on Arch Wiki and not on like all the other random shit they train it on) then it would be quite useful. But the thing is it isn't really profitable the way they think it will be. I'd compare it to like software in general.
I think we should push back on the AI term anyway. That's marketing BS. They are machine learning data processors. Their outputs are determined by their inputs. The viable economic strategy for them is to treat each LLM model more like a piece of software. You purchase X model and it does a specific thing. Or it comes bundled with something else. I think Apple will likely make a local model that runs on their Macs as a tech support helper at some point. And it'll just be trained on like all the issues that come up with the Mac and can download updates where if 1 person has an issue it'll update and suddenly all the machines will know about that issue. Or like how Photoshop has the "AI" tools that auto fill images and crop things for you.
Those are genuinely useful and don't take a data center to run.
What has basically happened is tech companies put spellcheck on crack out there as AI and are now trying to convince the world it can be the next stage of human civilization. When in reality it's best used as what it is. Spellcheck but for other things. You remember how back 10 years ago you had that thing where your phone would reccomend the next word for you to type and you'd click one of 3 or 4 options? That was an LLM. The tech isn't new. They just decided to pump so much data and energy and compute into it that it became sentient and it didn't work. So now they're trying to convince everyone it DID work so they don't go broke.
They do take a datacenter to train though, the model you're running was trained via looting the commons at staggering expense.
If you tried to train a model with less data, like just the arch wiki or something, it doesn't work. The architecture requires enormous amounts of training data to produce good results.
GPT2 ate 8 million webpages and
And the outputs are like a teenager on lsd
Well it already exists lol. Not using it isn't going to bring the energy it used to train back. That's a past cost. We can just... use the ones that already exist and tailor them for specific uses. Rather than continuing to train bigger and bigger generalist models. And I'd love to know if you have some sort of source for the less data not working thing. The amount of data used to train models varies wildly as far as I know. Why do you think specialist models trained on specific data sets wouldn't work?
It will become rapidly out of date as relevant text drifts from the training set.
It isn't some mind you can teach new things.
Actually you can add new data to its instruction set. If all you need it to do is know bash commands. You just have to dump the commands for the programs you have installed and add each command with a summary of what it does and the correct syntax to the instruction text. That doesn't require new training. And bash commands don't change that often. The need to constantly update is really only for the one you need to be generalist. If all you need is a bash helper what exactly are you retraining it for? Maybe each new Debian stable build you'd do an update to it?
I am referring to the neural network. Like if you want it to emit arguements for a bash command that will actually fix some software issue (and some other program in the stack then runs the command with the arguments appended) the ability to emit text which is plausible is based solely on correlates in the training data.
As software configuration, language style, relevant information etc drifts from the training set this becomes less likely.
If the tasks are simple and well understood an llm is overkill vs simpler software. Like if you're configuring something ansible or puppet are better choices, if you want to generate puppet config as the data plagurised in the training set becomes obsolete due to API changes performance will drop rapidly.
If you want to just fill out bash scripts use yaasnippet lol
Can you say to yaasnippet "what is that command that lets you control screen rendering settings?" and have it give you info on xrandr?
Yaasnippet is for writing software boilerplate via customisable macros. If you just want to look up documentation we actually have several tools for that already, which have the benefit of empowering you. (EDIT: this is too glib. I mean to say that documentation is more reliable, and because it is structured carefully by humans - some better at doing so than others admittedly - you pick up the overall structure and purpose of things. Computing stops being a set of weird specific magic spells and you start being able to solve your own problems. Or "are you saying I'll know how to use xrandr?" "I'm saying you won't have to")
However, what I am talking about is what happens when you start using a new piece of software that didn't exist or wasn't popular when the llm was trained? The information will be out of date and the answers not useful.
Also move off X lol. Wayland finally landed, it's actually good now. X11 is deprecated and only receiving maintenance patches. Automation is a bit fiddlier but the upside is a random piece of software can't send your entire screen to someone else.
I appreciate this perspective. It helps a lot. Ty