I know AI/LLM hate is strong here, so this is going to get some blow back. But there's a lot of Linux folk on here, so let me frame it this way....
My understand of the Linux/unix design philosophy is building small, efficient programs that do a limited set of tasks very well and that can be strung together with other programs that do other tasks very well. This is in opposition to the " be everything" program concept of Windows and Microsoft Office Suite. At least this is how would describe the difference to non technical friends: Nothing you think of as your OS in windows is actually what Linux is replacing. You're getting the Linux kernel packaged up in a distro that combines a bunch of smaller pieces (file explorer, window manager, etc) that you can still customize from there.
When I look at the approach to AI, I see the same thing. I've dabbled enough in ML/LLMs to know that LLMs are effectively very fancy next word predictors or for the case of image/video GenAI, next pixel predictors. As others have said countless times, there's no consciousness or understanding of the context, but you can ask it things in natural language and it will try to produce whatever you asked for in the same app regardless of context.
From a science project standpoint, this is cool, but it doesn't seem scalable or consistently reproducable and the energy use and easily found blunders seem to support that thought.
So, my question is why is no one building AI with a Linux philosophy? Small purpose built ML models with a language processing/triage model on top? Oh this person has a question about history, send them to the history module. This person wants to edit a photo, send them to the photo editing module. Then let those modules dig deeper from there. That's how we do customer service with real people after all. With this way we could refine each specialization individually instead of having a giant model that consumes tons of resources and is error prone.
If you'd like to analogize with special-purpose programs versus general purpose ones, you could consider an LLM to be more like an "operating system" rather than a single application running on that system. The LLM isn't specialized in a particular task, it supports running any task you want to throw at it.
In this analogy, the specialized applications that are running on the LLM would be the things called "agents" and "harnesses." They're the parts that hold the code that is tailored specifically to the particular task that they're for. So if I wanted to set up a system using an LLM to, for example, read court transcripts and automatically search legal databases for statutes and case law relevant to whatever's being discussed in them, I wouldn't simply copy and paste the transcript into the LLM's context and expect something useful to come back. I'd have to embed the LLM inside a system that prompts it correctly, has tools that can search legal databases, mechanisms for storing intermediate results, scripts to check the formatting of inputs and outputs, and so forth. None of that accessory stuff needs to be an ML model, it can just be conventional programming. Trying to train a specialized ML model from the ground up to do all that stuff without the associated harness helping it would be hugely wasteful.
I suppose another analogy could be the hardware itself. For every task that we use general-purpose von Neumann architecture CPUs for we could create a specialized chip on a purpose-specific circuit board. But instead for most tasks we find it's much cheaper and more convenient to create general-purpose computers and then program them for these specific tasks. Nowadays you'll often find that simple home appliances with just a few buttons and a few functions will have a full blown microcontroller inside them with firmware. Probably lots of unused inputs/outputs and ram and whatnot. That's because it's far cheaper and easier to build a factory that stamps out millions of general-purpose programmable chips than it would be to have hundreds of factory lines that each do a run of ten thousand custom-designed chips. It's genuinely less wasteful doing it that way.