I think it's fair to be nervous around people who conspicuously hide their identity. I've been mugged twice and in neither case did the person walk up to me and introduce themselves first. There's nothing wrong with giving people hiding their features a wide berth up to crossing the street, especially late at night. People who are just wearing what's comfortable will keep on going without interactions but if someone changes course to intercept you - especially hurrying to come up behind you - it's not at all racist to react defensively, it's just rational.

People can wear what they want, I will judge them (sort of) if they are obscuring their features but I do understand there are a lot of reasons for doing so (including not wanting a negative interaction yourself). I've had busty friends that adore their hoodies because it lets them avoid unwanted attention and cat calling - but it's equally fair to be concerned that someone obscuring their identity has a malicious intent.

Just like, be informed by attire but don't pass permanent judgement - and if you get mugged because you're helping someone that looks like they're in distress (points to self) it's okay - even if it fucking sucks you made the right decision.