Unless you're a ferengi. /s
I think a big part that's skeevy to me is that gender and sex are comparatively unimportant individual traits, referring to someone by their gender happens far more often for women and it's a hold over of misogyny. There are much more interesting individual traits that identify us than our sex or presented gender.