I was clearly an indoor house human and not allowed outside.
I read somewhere that scientists could prove Dogs know we are not dogs but Cats do not know we are not cats; to cats we are just weird looking cats that are very generous as we constantly share "our" food
So to your cat you are just a weird indoor cat! And they can tell who is allowed out and who is not because we have 3 cats and only 1 is allowed out (long story). When she is coming back, another one of our cats eagerly awaits and sometimes tries to poker head out and our outdoor cat will scold and hiss at her until she comes back into the house completely
This behaviour is so strong that the curious indoor cat got used to sniffing the outside air from under the open door (instead of just peaking out) to avoid the scolding