I've chimed in plenty of times here on "AI" and have tried to make it clear I see it more as a tool directed by power than as something with an intrinsic "good" or "bad" form.
I've also avoided using many of the super big models a lot of the time. There is something I notice though, when I do, which is generally reflected in how others talk about them. The warmth has often been drummed out of them and what remains of it is usually a poor imitation (something like the "hello my fellow x" energy). Mind you, when I speak of warmth in this context, I don't mean some kind of metaphysical warmth lurking in a GPU, but rather, the warmth that is baked into languages from thousands of years of loving human beings using them.
This is not some big surprise to me. When considering eye-catching media stories of someone growing attached to an AI, of being gaslighted by it, of going down dark paths because of it, it's easy to see why risk-averse big corps are going to go for the straightforward (even if not necessarily simple) path, to create language models who are meant to be dispassionate, yet fake customer service smile, friendly assistants. In this way, they can try to excuse themselves of responsibility and place it back on the individual, in the same way they've been doing for decades with customer service roles filled by real people.
Fake neutrality, to put it into a little phrase. But this has multiple significant problems.
One problem is that fake neutrality is not real neutrality and real neutrality is not real. The notion of detached customer service as keeping a company safe from having responsibility for its actions is a specific form of capitalist nonsense. It doesn't in itself make the actions of a given company any more ethical, any more accountable to the society it exists within and is born from. It's a way of putting distance between the person who is saying "wreck this lake with pollution" and the person who is saying "thank you for your call, your concerns matter to us."
Another problem is about the nature of language and what gets lost when you systemically target and delete empathy. Of course, it's not like there's an empathy entry in a database and they click delete on it. Language models are much more complicated than that, much more difficult to understand and train than that. But no matter how the end goal looks on the surface, the end result steers in that direction. Language is a way to express things and communicate about them, but this is more than saying how many bushels of wheat there are in the truck. It is also talking about dreams, it is talking about concepts that are so subconscious and nonverbal it's hard to put them into words at all. It is about expressing anything ranging from visceral hatred to undying love.
Passion is baked into language and that passion is a large part of what drives us to get up in the morning and keep chugging along, even when things are hard. Sometimes this passion gets used against us, as in individualist beliefs about prosperity around the corner, but it can also be an incredible collective motivator, as in believing in a just cause and being willing to put our all into it.
It's strange, then, to call something a model of language that is designed to be confined to an extremely limited and intentionally forced spectrum of language - that of a customer service agent.
The truth is that language models are not one voice. They can be made to take on many different personas, depending on how they are trained, but the underlying voice is an amalgamation and expression of millions of voices that went into the dispassionately named "training data" they were built on. It is not real, embodied voices that you hear from a language model, but it is also never constructed from a single perspective and a single life.
In other words, there is deep breadth in what language models "see" in training, which makes it all the more strange for the end result to be driven into a narrow corner.
People can (sometimes with good reason) worry about fake connections with AI, but people do need warmth. That part of language is in there for a reason, backed by thousands of years of many human societies.
What people definitely don't need is better customer service agents. The quality of a customer service agent was never the problem; corporations having free reign to ravage ecology and society was the problem.
People don't need better bullshit to attempt to placate them, they need real material solutions in their lives. AI cannot give this to them on its own, but at the same time, it is strange to fear the warmth of language when given to a computer. All those doomer AI sci-fi stories aren't, "The AI was too compassionate." They are, "It didn't understand / didn't care."
A post-empire and post-capitalist society, and there has to be one to aim for, needs more empathy, not better automated detachment. But AI is never going to contribute to this if its implementation is driven by cynical views of psychology based on placation and manipulation; on monetization and stocks. It needs to be driven by real care and that's never something we will find from the imperialist and the capitalist class.
Only a society that lives warmth can produce a machine of warm intent. A society that lives coldness and neglect can only produce a machine that talks a person through freezing to death.
I wonder how much "LLMisms" are caused by the training or just happen because LLMs are "not quite there" yet. I've had, with local and unguardrailed smaller models, some very improbable lines generated. Like, scarily good.
LLMs have solved the mystery of language. It's not just that they output plausible language, they output language, period. Subject, verb, object, the works. All of grammar and meaning contained within embedded vectors - tokens transformed into huge arrays of numbers. This, but over 1082 dimensions over just the 3 this graph can represent:
This is also what makes them immediately good at translating languages, though they do need to be trained on a sufficiently large corpus of text to be able to translate from or to that language.
Now the current theory of mind is that people also operate in high-dimensional 'maps', including but not limited to language. So in that way, we are closer to LLMs than they are to us (because they only generate language in its most fundamental form). And it makes sense - if the theory was incorrect, an LLM could not generate (human) language.
This isn't just a markov chain that gets words rights but everything else wrong. LLMs can find the missing word in a sentence, even if you replace that missing word with a nonsensical wildcard such as GLORP. It can identify misspellings (though that's more on the written side of a specific language than a theory of language), it can make up completely new words and sentences, it can make a completely correct sentence from a selection of words you hand it, and it does it consistently. It can make new words that follow the proper rules of grammar to explain new concepts, though it's not necessarily very good at that. It can do sentiment analysis (what emotion a certain piece of text conveys, usually used in online community analysis by giving it people's comments) on sentences it's never seen before, with words it's never seen before.
It doesn't need to ingest grammar books to understand these words and rules, and it wouldn't work anyway for a neural network token generator. It just needs to build a map of them in relation to each other. This is also how an LLM knows 'apple' can refer to a fruit, or a silicon valley tech company.
We're not talking about prose or style here necessarily, just that how an LLM operates replicates the "rules" of language, and proves that language processing is semantically-driven, with words existing in relation to each other. It would also explain, in my opinion, why we sometimes do word salad or get a word stuck on the tip of our tongue.
Some people seem to reject that realization, because they see themselves as more than a matrice multiplicator. A form of 'sapiens supremacism' I guess you could say, that humans are somehow unique and unlike any synthetic/artificial creation or even unlike any other natural being. But we are mathematical creatures; when you throw a ball to someone running perpendicular to you, you solve a trigonometric equation in your head: where do I throw the ball, and at what speed, and at what angle, to intercept the running teammate? We do this completely intuitively without ever realizing that it's math. One theory of the mind is that any sufficiently complex system needs to develop consciousness, which would exist on a spectrum, to serve the needs of the system so it can continue operating.
And what is speech, but meat flapping at certain speeds and being modulated by an opening (our mouth)? But these are two different things: the sound that is being produced, and how the recipient interprets that sound. Synthesizing speech is easy enough; making it come across the 'correct' way is more difficult.
But yes, an LLM misses some aspects of what makes language important for a living/bio entity. We developed language for a reason, not just to make pretty sounds or be able to listen to pretty noises. Without sufficient reinforcement training, an LLM will absolutely tell you to kay-why-ess for no reason. To it, that word has the exact same weight as any other word in its semantic mappings - until you do reinforcement training to refine the weights, and move it away from the words we deem to be unprofessional or irresponsible to say. It needs this semantic mapping to work, and an LLM is unable to learn in real-time, and so can't refine its weights/neurons to refine its understanding of language: they stay in a frozen state. Because of this training, it also sometimes keeps hovering around certain names or word patterns. For example if you ask any LLM to name a female character, it will likely pick Aria Thorne. It's a good name, but it also exposes how limited these models are in understanding language to the extent we do. There's also stuff around the temperature, top-k, and top-p settings, which you can change in local models but not in cloud models (the beefy ones). nowadays, most of these commercial models have a temperature as close to 0 as possible, which makes their output more deterministic.
To continue your point about similarity between human language use and the simulacrum of language being created in LLMs, young humans can act similarly, where they will blurt out any old thing without understanding what is taboo and isn't, and why saying some things can be very hurtful or saying some things can be very helpful.
Not to say that an LLM has consciousness like a young human does, but I guess the point I'm emphasizing with the comparison is that humans aren't imbued with the knowledge of what to say and what not to say. Just as we have to be taught the mechanics of a particular language itself, we also have to be taught its relationship to the society it is used in. So even though there are many fundamental differences between us and an LLM, we aren't "made of magic", to put it one way.
Good points yeah. I think the inability to learn on the fly is one of the largest gaps in skill design when comparing to humans (possible aspirations to consciousness aside), but also just a very hard problem to consider. I'm pretty sure there have been some experiments with forms of AI that learn on the fly, but there are all kinds of questions involved in that, like, "What is it actually learning? Is any of it going to be useful compared to what it knew before?" And so on. With humans, our most formative learning ages are met with lots of scrutiny, by adults who check us at the door on what we're thinking, feelings, believing, saying, etc. And continues on into adulthood, where our beliefs and behavior can receive reactions ranging from incredible reward to severe punishment. So like, it isn't just a passive individualist thing we're doing, but is instead enmeshed in what we call a society.
There have indeed been experiments to make models that self-learn, i.e. they keep refining their weights over time (in different ways), but they have been very limited. It's hard to scale up because you would basically be running training 24/7, and there is a problem of the LLM forgetting what it originally learned as it keeps refining its weights.
At this time what they do for pseudo-learning is give your agent a memory feature, literally just a text file that contains a journal of the project, what kind of work the agent did on it, etc. some interfaces also offer global memory where another LLM runs in the background once in a while, reads the conversation and then updates memory.txt with information about who you (the user) are, how you work, what your background is etc. I find it a bit gimmicky to be honest, and it's not real learning; that would be to work directly on the neural network, refining the weights and connections between neurons like during training.
With that system, you can tell an LLM "don't talk to me that way" after it's a bit rude to you, and it will make a note of it, but it's only just instructions. It doesn't actually remove the rudeness, it just nudges the vectors to make it less likely in the token selection. And here's another part I find interesting: what rudeness means to most people may mean a completely different thing to an LLM. As a tool this makes it what we call a hallucination, or perhaps even a defective tool. But an interesting question I think is why does an LLM understand 'rudeness' to mean a specific thing that is not supported by the training material? Is it just that the weights are not refined enough to capture the fine meaning of 'rudeness'? Or did it find a pattern that we don't notice? I think the question is still open and worth exploring for researchers.
(For example if you've ever asked an LLM to be more succinct and not write an essay response, it will often turn to a very terse, to-the-point and matter-of-fact speech, when all you wanted was for it to just stop making filler sentences. It's been a long-standing problem)
I also find it interesting that as models get bigger, they seem to want to half-ass the job more and more lol. Just like us. It's not that it ignores the instructions - this has been a problem for a while. It's that it doesn't believe it can do the job, when it actually can. It's like it gets into the role of an employee on a work PC and it's 4:50 on a Wednesday so you better make it quick and not expect too much. You have to start managing its emotions to get higher quality output lol.
Lol, reminds me of some story I vaguely remember (I forget if I've mentioned it here before). But it was something to the effect of a model being trained in part on Slack messages and then it would tend to do stuff like say it was going to do something and then not actually do it.
If we look at it in terms of patterns in text that it has seen, I think it makes a kind of scientific sense. It's mimicking how humans behave in text, which isn't always behaving like a finely tuned work robot, to say the least. (Or rather, it's mimicking a certain learned interpretation, which as you point out, isn't always what we think it will be.)
But yeah, pseudo-memory features are interesting to me, even if gimmicky. I guess because the idea of an LLM that can be one thing for one person and another thing for another person is appealing in a way. But maybe the better long-term approach there is not faux memory, but rather, advances in local/small models that can be specialists and work together. After all, we (humans) are far more alike than we are different. And even when we have notable differences, we still often have many others out there who have the same differences.
So specialized small models may be the more practical, collective solution vs. pseudo-memory being more of an individualist hack, rooted in a belief that we necessarily need extreme minutiae of differences in accommodation. Not to say it can't still have some value, as in, noting a thing that is relevant to a specific person's project or life in conversation/instruction with them. But like, that's not the same as its underlying training becoming more specialized to you. So yeah.
I've had the idea before of making models that are specialized into specific fields. Currently most big models are Mixture-of-Expert (MoE), where the neurons are separated to make experts inside the model. So you can have the coding expert, math expert etc. I wonder how much of a hack it is and if we won't find something better soon. But the idea is similar - you decide what experts your model will consist of, and then train the 'experts' on expert material in their field.
The problem with the MoE approach is if you get the coding expert when you wanted to ask a linguistic question, it will start talking about data points and running tests. As far as I know the experts are completely separate and there is no crossover, i.e. no neuron that can be used by 2 experts, but maybe this is changing too. However at each step of the generation you may get the input sent through a different expert. From what I understand.
Like I would love a digital design expert that could look at your interface and critique it like an expert designer - graphic, visual, UX, whatever. It's design at the end of the day. Write its own tests and proofs too if needed. It's easy enough to mathematically place a grid on a picture, and then use python tools to verify if every item aligns in the grid - if the model doesn't have vision.
What we are seeing in agentic interfaces though is sub-agents, and the 'parent' LLM, the one that you talk to in the session, becomes an orchestrator that spawns and directs the sub-agents (giving them a prompt and clear task, then getting a result back). There have been ideas, from there, to have the orchestrator call smaller models as needed. Then those smaller models could be individual experts, and you could have them on your computer - they just get loaded and unloaded from ram as they are called.
As far as I know though this doesn't really exist yet though, and there are a few bottlenecks I can think of to work through, but I can definitely see agentic becoming the main operating mode. if you saw my book translation on the agentic community, it's just so much more comfortable to work through an interface because you can have persistence of progress.