"most people" are (1) a much smaller group that you think and (2) wrong.
If your behavior can at all be described as sexist, then you're not a feminist. That includes both Feminism Appropriating Radical Transphobes like JK Rowling.and outright sex pests like Andrew Cuomo.