In my country voting is anonymous, how would anyone know based on their sex or color or whatever how anyone voted. Here, your vote is considered private so asking a person how they voted as they walk out the polling station is just a foreign concept to me.
What's even weirder is how you came to your conclusions. You really have to explain the mental gymnastics you used, because you're really coming out of nowhere.