How is this thought experiment different from Platos allegory of the cave?
Not sure if I follow. My point with the Chinese Room experiment focus on the idea that the "machine can't have a mind", and that even without a mind it can "display intelligence" (even without it). Almost like the semantics over concept.
While Plato is about the real world and the projected world. Not sure if they are the same, but can be close enough.
Ps.: Also, I'm not super familiar with the Plato Cave, I know about it, but didn't read the originals like most in this specific place.
I can directly observe my own thought processes because I am inside the system generating them. You cannot [...]
Totally agree with you.
But when you are bounded by a rulebook, the "thought processes" is irrelevant, point of the post.
But for the observer outside of the box it will still look like "magic", or "intelligence" when there is nothing intelligent there, just rules.
My point is, even if the machine does have a inner person, it yet don't use this to communicate, since it only use the manual to translate symbols that carry meaning to us, but no meaning to the machine. And thus the agency that I want the companies behind the machine to have it, instead of "oops the machine did, we didn't knew about it, sowwy"
But I may have misunderstood your point in this whole thing.