A lot of the reason why "Female" has a bit of a negative slant, is because of the kinds of people/communities that overused the word.

Those groups used female as a way to say that women are only useful as somewhere to put your dick. There didn't really seem to be a group using male in a dehumanizing way, so it doesn't really have the same negative feeling.

Kinda like how if someone just comments "Jew" on a post it can feel negative, but if they say "Canadian" or "Bulgarian" it feels neutral.