I can only assume you're not in the US. I'm in Aus, I don't think there's any rule about political party branded clothing here but I think if you were wearing clothing with a political parties logo and not directly involved in the party, either an actual politician or helping with their campaign, then people would think there was something dodgy going on ..