Did they specifically ask people about him (like "look at this guy, do you know who that is?"), or did they just assume because no one approached him? I couldn't imagine actually bothering someone in public just because they're famous. Though I also wouldn't trust in my ability to recognize a celebrity, I'd definitely not be the one to figure out that Clark Kent is Superman if I was living in that universe.