How is this thought experiment different from Platos allegory of the cave?
I.e. "if someone grew up in a cubicle..."
So the answer is, perhaps surprisingly to some, yes.
I can directly observe my own thought processes because I am inside the system generating them. You cannot; you can only infer my intelligence from my behavior. Likewise, an intelligence instantiated inside a machine may have first-person access to its own cognition, while an outside observer can only infer intelligence from its outputs and mechanisms.