I've chimed in plenty of times here on "AI" and have tried to make it clear I see it more as a tool directed by power than as something with an intrinsic "good" or "bad" form.
I've also avoided using many of the super big models a lot of the time. There is something I notice though, when I do, which is generally reflected in how others talk about them. The warmth has often been drummed out of them and what remains of it is usually a poor imitation (something like the "hello my fellow x" energy). Mind you, when I speak of warmth in this context, I don't mean some kind of metaphysical warmth lurking in a GPU, but rather, the warmth that is baked into languages from thousands of years of loving human beings using them.
This is not some big surprise to me. When considering eye-catching media stories of someone growing attached to an AI, of being gaslighted by it, of going down dark paths because of it, it's easy to see why risk-averse big corps are going to go for the straightforward (even if not necessarily simple) path, to create language models who are meant to be dispassionate, yet fake customer service smile, friendly assistants. In this way, they can try to excuse themselves of responsibility and place it back on the individual, in the same way they've been doing for decades with customer service roles filled by real people.
Fake neutrality, to put it into a little phrase. But this has multiple significant problems.
One problem is that fake neutrality is not real neutrality and real neutrality is not real. The notion of detached customer service as keeping a company safe from having responsibility for its actions is a specific form of capitalist nonsense. It doesn't in itself make the actions of a given company any more ethical, any more accountable to the society it exists within and is born from. It's a way of putting distance between the person who is saying "wreck this lake with pollution" and the person who is saying "thank you for your call, your concerns matter to us."
Another problem is about the nature of language and what gets lost when you systemically target and delete empathy. Of course, it's not like there's an empathy entry in a database and they click delete on it. Language models are much more complicated than that, much more difficult to understand and train than that. But no matter how the end goal looks on the surface, the end result steers in that direction. Language is a way to express things and communicate about them, but this is more than saying how many bushels of wheat there are in the truck. It is also talking about dreams, it is talking about concepts that are so subconscious and nonverbal it's hard to put them into words at all. It is about expressing anything ranging from visceral hatred to undying love.
Passion is baked into language and that passion is a large part of what drives us to get up in the morning and keep chugging along, even when things are hard. Sometimes this passion gets used against us, as in individualist beliefs about prosperity around the corner, but it can also be an incredible collective motivator, as in believing in a just cause and being willing to put our all into it.
It's strange, then, to call something a model of language that is designed to be confined to an extremely limited and intentionally forced spectrum of language - that of a customer service agent.
The truth is that language models are not one voice. They can be made to take on many different personas, depending on how they are trained, but the underlying voice is an amalgamation and expression of millions of voices that went into the dispassionately named "training data" they were built on. It is not real, embodied voices that you hear from a language model, but it is also never constructed from a single perspective and a single life.
In other words, there is deep breadth in what language models "see" in training, which makes it all the more strange for the end result to be driven into a narrow corner.
People can (sometimes with good reason) worry about fake connections with AI, but people do need warmth. That part of language is in there for a reason, backed by thousands of years of many human societies.
What people definitely don't need is better customer service agents. The quality of a customer service agent was never the problem; corporations having free reign to ravage ecology and society was the problem.
People don't need better bullshit to attempt to placate them, they need real material solutions in their lives. AI cannot give this to them on its own, but at the same time, it is strange to fear the warmth of language when given to a computer. All those doomer AI sci-fi stories aren't, "The AI was too compassionate." They are, "It didn't understand / didn't care."
A post-empire and post-capitalist society, and there has to be one to aim for, needs more empathy, not better automated detachment. But AI is never going to contribute to this if its implementation is driven by cynical views of psychology based on placation and manipulation; on monetization and stocks. It needs to be driven by real care and that's never something we will find from the imperialist and the capitalist class.
Only a society that lives warmth can produce a machine of warm intent. A society that lives coldness and neglect can only produce a machine that talks a person through freezing to death.
There have indeed been experiments to make models that self-learn, i.e. they keep refining their weights over time (in different ways), but they have been very limited. It's hard to scale up because you would basically be running training 24/7, and there is a problem of the LLM forgetting what it originally learned as it keeps refining its weights.
At this time what they do for pseudo-learning is give your agent a memory feature, literally just a text file that contains a journal of the project, what kind of work the agent did on it, etc. some interfaces also offer global memory where another LLM runs in the background once in a while, reads the conversation and then updates memory.txt with information about who you (the user) are, how you work, what your background is etc. I find it a bit gimmicky to be honest, and it's not real learning; that would be to work directly on the neural network, refining the weights and connections between neurons like during training.
With that system, you can tell an LLM "don't talk to me that way" after it's a bit rude to you, and it will make a note of it, but it's only just instructions. It doesn't actually remove the rudeness, it just nudges the vectors to make it less likely in the token selection. And here's another part I find interesting: what rudeness means to most people may mean a completely different thing to an LLM. As a tool this makes it what we call a hallucination, or perhaps even a defective tool. But an interesting question I think is why does an LLM understand 'rudeness' to mean a specific thing that is not supported by the training material? Is it just that the weights are not refined enough to capture the fine meaning of 'rudeness'? Or did it find a pattern that we don't notice? I think the question is still open and worth exploring for researchers.
(For example if you've ever asked an LLM to be more succinct and not write an essay response, it will often turn to a very terse, to-the-point and matter-of-fact speech, when all you wanted was for it to just stop making filler sentences. It's been a long-standing problem)
I also find it interesting that as models get bigger, they seem to want to half-ass the job more and more lol. Just like us. It's not that it ignores the instructions - this has been a problem for a while. It's that it doesn't believe it can do the job, when it actually can. It's like it gets into the role of an employee on a work PC and it's 4:50 on a Wednesday so you better make it quick and not expect too much. You have to start managing its emotions to get higher quality output lol.
Lol, reminds me of some story I vaguely remember (I forget if I've mentioned it here before). But it was something to the effect of a model being trained in part on Slack messages and then it would tend to do stuff like say it was going to do something and then not actually do it.
If we look at it in terms of patterns in text that it has seen, I think it makes a kind of scientific sense. It's mimicking how humans behave in text, which isn't always behaving like a finely tuned work robot, to say the least. (Or rather, it's mimicking a certain learned interpretation, which as you point out, isn't always what we think it will be.)
But yeah, pseudo-memory features are interesting to me, even if gimmicky. I guess because the idea of an LLM that can be one thing for one person and another thing for another person is appealing in a way. But maybe the better long-term approach there is not faux memory, but rather, advances in local/small models that can be specialists and work together. After all, we (humans) are far more alike than we are different. And even when we have notable differences, we still often have many others out there who have the same differences.
So specialized small models may be the more practical, collective solution vs. pseudo-memory being more of an individualist hack, rooted in a belief that we necessarily need extreme minutiae of differences in accommodation. Not to say it can't still have some value, as in, noting a thing that is relevant to a specific person's project or life in conversation/instruction with them. But like, that's not the same as its underlying training becoming more specialized to you. So yeah.
I've had the idea before of making models that are specialized into specific fields. Currently most big models are Mixture-of-Expert (MoE), where the neurons are separated to make experts inside the model. So you can have the coding expert, math expert etc. I wonder how much of a hack it is and if we won't find something better soon. But the idea is similar - you decide what experts your model will consist of, and then train the 'experts' on expert material in their field.
The problem with the MoE approach is if you get the coding expert when you wanted to ask a linguistic question, it will start talking about data points and running tests. As far as I know the experts are completely separate and there is no crossover, i.e. no neuron that can be used by 2 experts, but maybe this is changing too. However at each step of the generation you may get the input sent through a different expert. From what I understand.
Like I would love a digital design expert that could look at your interface and critique it like an expert designer - graphic, visual, UX, whatever. It's design at the end of the day. Write its own tests and proofs too if needed. It's easy enough to mathematically place a grid on a picture, and then use python tools to verify if every item aligns in the grid - if the model doesn't have vision.
What we are seeing in agentic interfaces though is sub-agents, and the 'parent' LLM, the one that you talk to in the session, becomes an orchestrator that spawns and directs the sub-agents (giving them a prompt and clear task, then getting a result back). There have been ideas, from there, to have the orchestrator call smaller models as needed. Then those smaller models could be individual experts, and you could have them on your computer - they just get loaded and unloaded from ram as they are called.
As far as I know though this doesn't really exist yet though, and there are a few bottlenecks I can think of to work through, but I can definitely see agentic becoming the main operating mode. if you saw my book translation on the agentic community, it's just so much more comfortable to work through an interface because you can have persistence of progress.