The funniest exercise is to ask any LLM for a short story and they all come up with Elias the lighthouse keeper visited by Maria or some variation on that theme. That's not creative or conscious.

Not only that, they only act when directed. There's zero idle "thought" when they're not actively performing a task.

This is strictly false, generalization in LLMs is well documented.

It's also not relevant. Read the first point in the article: intelligence != consciousness. Individual ants, bats, and fish do not do anything very creative (certainly less creative than gpt-2), yet they are probably all conscious.

That's not the reason, we're in the same boat when it comes to knowledge in many ways.

But we don't even know what consciousness is so it's a waste of time reading the "3 reasons (*potential)" IMO.

Case in point have they ever turned on a model and left it for it to, on its own, decide "wait what's going on, who am I, what/where am I?". And start independently exploring?

Highly doubt it. If so, we're cooked

Also an LLM only produces output when it is prompted to do so. Between prompts it's not sitting there - waiting, wondering about the meaning of life, getting bored - until you prompt it again. It's basically dead.

I like to compare llms to a Galton Board. Instead of the regular layout of pins someone added or removed pins so the marbles on the bottom fall into nice to look patterns.

By identifying"memories" they can include the information in the context of the next update - this is akin to our short term memory. Make it self pruning, where the data is saved locally but not included in the next context, and it acts the same as own memory system.