for most of the history of the word, "man" has generally meant all humanity, or any human.

male men were called something like "wer", which survives in "werewolf".