Legally, a lot of places it's been court tested, and you actually can. Culturally, it's just kind of that way. Hair is banned in Arabia, tits can hang out in Papua New Guinea. The West says no boobs but hair is fine.

A genital covering is the only cultural universal, although it can sometimes be overly revealing by Western standards, like with penis gourds.