I've chimed in plenty of times here on "AI" and have tried to make it clear I see it more as a tool directed by power than as something with an intrinsic "good" or "bad" form.

I've also avoided using many of the super big models a lot of the time. There is something I notice though, when I do, which is generally reflected in how others talk about them. The warmth has often been drummed out of them and what remains of it is usually a poor imitation (something like the "hello my fellow x" energy). Mind you, when I speak of warmth in this context, I don't mean some kind of metaphysical warmth lurking in a GPU, but rather, the warmth that is baked into languages from thousands of years of loving human beings using them.

This is not some big surprise to me. When considering eye-catching media stories of someone growing attached to an AI, of being gaslighted by it, of going down dark paths because of it, it's easy to see why risk-averse big corps are going to go for the straightforward (even if not necessarily simple) path, to create language models who are meant to be dispassionate, yet fake customer service smile, friendly assistants. In this way, they can try to excuse themselves of responsibility and place it back on the individual, in the same way they've been doing for decades with customer service roles filled by real people.

Fake neutrality, to put it into a little phrase. But this has multiple significant problems.

One problem is that fake neutrality is not real neutrality and real neutrality is not real. The notion of detached customer service as keeping a company safe from having responsibility for its actions is a specific form of capitalist nonsense. It doesn't in itself make the actions of a given company any more ethical, any more accountable to the society it exists within and is born from. It's a way of putting distance between the person who is saying "wreck this lake with pollution" and the person who is saying "thank you for your call, your concerns matter to us."

Another problem is about the nature of language and what gets lost when you systemically target and delete empathy. Of course, it's not like there's an empathy entry in a database and they click delete on it. Language models are much more complicated than that, much more difficult to understand and train than that. But no matter how the end goal looks on the surface, the end result steers in that direction. Language is a way to express things and communicate about them, but this is more than saying how many bushels of wheat there are in the truck. It is also talking about dreams, it is talking about concepts that are so subconscious and nonverbal it's hard to put them into words at all. It is about expressing anything ranging from visceral hatred to undying love.

Passion is baked into language and that passion is a large part of what drives us to get up in the morning and keep chugging along, even when things are hard. Sometimes this passion gets used against us, as in individualist beliefs about prosperity around the corner, but it can also be an incredible collective motivator, as in believing in a just cause and being willing to put our all into it.

It's strange, then, to call something a model of language that is designed to be confined to an extremely limited and intentionally forced spectrum of language - that of a customer service agent.

The truth is that language models are not one voice. They can be made to take on many different personas, depending on how they are trained, but the underlying voice is an amalgamation and expression of millions of voices that went into the dispassionately named "training data" they were built on. It is not real, embodied voices that you hear from a language model, but it is also never constructed from a single perspective and a single life.

In other words, there is deep breadth in what language models "see" in training, which makes it all the more strange for the end result to be driven into a narrow corner.

People can (sometimes with good reason) worry about fake connections with AI, but people do need warmth. That part of language is in there for a reason, backed by thousands of years of many human societies.

What people definitely don't need is better customer service agents. The quality of a customer service agent was never the problem; corporations having free reign to ravage ecology and society was the problem.

People don't need better bullshit to attempt to placate them, they need real material solutions in their lives. AI cannot give this to them on its own, but at the same time, it is strange to fear the warmth of language when given to a computer. All those doomer AI sci-fi stories aren't, "The AI was too compassionate." They are, "It didn't understand / didn't care."

A post-empire and post-capitalist society, and there has to be one to aim for, needs more empathy, not better automated detachment. But AI is never going to contribute to this if its implementation is driven by cynical views of psychology based on placation and manipulation; on monetization and stocks. It needs to be driven by real care and that's never something we will find from the imperialist and the capitalist class.

Only a society that lives warmth can produce a machine of warm intent. A society that lives coldness and neglect can only produce a machine that talks a person through freezing to death.