The argument predates LLMs by decades, and in other forms it has existed for centuries. It doesn't directly corellate to modern LLMs. It is an example of how a mechanism could mimic human intelligence without actually being intelligent. It's a thought experiment. LLMs are not lookup tables, they are neural network based statistical models, but a similar argument can be made.

In the second scenario, there is no “native speaker” that is telling you what to write. There is a list of instructions that you are following that is the same as the instructions that the computer was following with the first scenario.

The argument is that if, by following the exact instructions that the computer is following, you still do not understand the Chinese conversation, why would you say that the machine understands?

I haven’t thought about the idea enough to say I agree or disagree, but I hope I understood your comment and that the explanation helps.

A lot of things seem trivial with 50+ years of hindsight.

Even then, most people right now still don't grasp the nuances of the debates around machine intelligence that have been occurring for the past hundred years.

I think you grasp the point, it is exactly what an LLM does

🤷 🐈‍

You have a cat. The cat meows at you. You do nothing. The cat keeps meowing at you. You offer it food as a guess. The cat stops meowing.

The next day, the cat starts meowing at you again, but slightly different. You give it food. It keeps meowing. You give it a belly rub, the cat stops meowing.

You have inadvertently learned the correct response to two types of cat meows. However, you do not speak cat. Nor does the cat speak human. You are just giving a response it expects for a given input.

If you got a spare minute and then 600 of those you could read Peter Watts Blindsight, the main viewpoint character is basically a chinese room which is not a spoiler but an important keypoint of the whole story. Its also sci-fi and a bit weird

While we are at it, there is this page that is an interesting display of the making of a fan movie trailer https://blindsight.space/

Yeah. It basically states that because all your neurons and synapses in the human brain follow quantum physics and can be simulated by a formal program, and do not know what they are doing or know meaning, humans are not sentient but merely simulate being a mind.

It's really infuriating how one of the dumbest and self evident fallacious thought experiments gets so famous. Because idiots keep repeating it like the sub-sentient LLMs they are.

Because that is what we are really learning from LLMs. It's not shocking that LLMs aren't sentient, it's shocking that we now know what minds can talk seemingly intelligent without having to be sentient. Which explains the current state of the world.