As a man who owns multiple kilts I can confirm there will be some assholes that have something to say.
On the other hand, those encounters have been outweighed by the compliments and enthusiasm I've also gotten.
All that said, I say just do you. Haters gonna hate, people will judge. Stop caring about those people and live Your life.