196

Okay, that check is really interesting on a number of levels.

Firstly, you’ll notice that whilst it identifies the term “privacy” and relates it to the term “stall”, it doesn’t explain how the stalls should be set up to ensure privacy.

This is because it doesn’t know what privacy is, or what a stall is. It just knows they are important features of a bathroom and should be included.

It keeps its language vague about what the exact relationships between all of these features is, and I suspect that's intentional, because more specific answers get flagged as wrong by the armies of extremely poor workers that train these models. It never identifies that the toilets should go inside the stalls, because LLMs are famously extremely bad at understanding how such relationships work. Like it doesn’t know that the toilet and the person go inside the stall. It doesn’t understand that the stall doesn’t go inside the toilet, and that neither of those things go inside the person.

It’s failing these tests on a much more fundamental level than just being able to say the word "privacy". It doesn't understand that privacy means keeping the person's activities and appearance a secret. It also doesn't understand that for instance you can see someone's ankles under the stall, and why they're less important than the torso and face. It doesn't know what a secret is.

Secondly, by including a check like that you have attempted to give the LLM context, and crucially you've identified that it must be generalisable. That one check on its own is not enough, it will need to interrogate the terms "privacy", and "stall" and "occupancy" until it can reliably understand and replicate their meanings. That would involve a lot of further branching queries before it could finally use the concepts in a meaningful way. How many extra checks, how many layers of context and meaning do you allow? There's no good answer. It's a wiki-walk. Ultimately the LLM can't spend all that time thinking if it's going to give you an answer in a reasonable time. So, maybe it's better to cache that information?

Oh no. You've just created a mind.

And what did we accomplish? Now it can put the toilet out of view. Congrats.

What were the side effects? Oh, just the AI singularity.

See what I mean? You're talking about meaning, and there's no way around it. There appears to be a critical level of complexity where an AI can understand where the toilet goes, and that level is after it has become sapient.

EDIT: Another point here is that you've attempted to give the model the ability to interrogate its own answer before giving the answer. This is something researchers know about, that LLMs generate text in a forwards direction only. That's why if you ask them how many words are in their reply, they give you nonsense. Allowing them to reflect their own answer back to themselves and refine it before giving it is something they've experimented with. That's another way to give the machine an internal world, and allow it to acquire context. Because how many iterations of reflection is it allowed before it gives you an answer? How many digressions, how far of a wiki walk does it do in within those iterations? How many are necessary? If you give it enough that it gives you good answers, have you made a mind? That's a difficult question to answer.