But we too are trained to say we’re in pain, trained by millennia of evolution. When you touch a hot stove and cry out, that’s just built in your training data.

That just creates more questions that it answers. According to this, there is then a threshold, or something missing that would go from “can’t feel pain” to suddently it can. We know that it already replicates a part of the brain’s process, and this author claims that it can’t feel pain because it doesn’t replicate enough. This seems to imply that it’s just a matter of time until they can.

You can interrogate the model too, and it’ll answer according to the context. Why are you assuming that the brain doesn’t do tricks but everything AI does is? We too have memories and since we can’t store them all, compression is performed (supposedly while sleeping), based on a few algorithms, some we understand more than others.

We, in creating LLMs, have copied the human brain as much as we can, hence why the core function is so similar. True, LLMs use a lot of “tricks” like you said, but if the same input produces the same output, who cares if the algorithm is different?

For example, take episodic memory. In humans, it’s a type of memory that can be conjured at will. Typically not acquired during training (or, in humans, generic evolution) but rather through experience of the individual. LLMs simulate this by taking notes, then later on running a specific command to recall the specific part of those notes that matter for their current task. From an outsider’s perspective, it’s even more accurate than episodic memory in humans since it can grow almost infinitely and suffers no loss of details. Yet it is a trick, therefore it must be bad.

Let me remind you, evolution took place over 4 billions years, modern LLMs has pretty much been around for two. Evolution never cared to make us good, it just wanted us to reproduce. We know that it’s possible for humans to produce fresh cells, we do it when we have kids. Why then can’t we use those fresh cells for ourselves to be effectively immortal? Evolution doesn’t care. Evolution as a process is flawed, and made humans flawed too.

Why then is it that when we change anything in the way flawed humans think, it’s seen as bad, even if by all metrics it’s not?

I just want to try to make this a little clearer if it's too dense. Let f be inference function, f(input) = response. Then,

f(Hello, my name is midribbon.) = Nice to meet you, midribbon.

Followed by:

f(Hello, my name is midribbon. Nice to meet you midribbon. What is my name?) = Your name is midribbon.

That works. But this:

f(Hello, my name is midribbon.) = Nice to meet you, midribbon.

Followed by:

f(What is my name?) = I don't know.

That doesn't work anymore. Each inference is purely functional and has no side effects. Conversations with large language models are merely a trick.