If I went up to someone wearing a pride flag and tried to relate to them over that flag, only to find out that they're just an ally wearing it...

Hang on let me rephrase that

If I found out that an ally (depending on the area they're in) risked their lives in order to make it more difficult to identify my minority group...

I don't think I'd mind. I would love to see cis people walking around in trans pride gear with the phrase "aren't we all people?" All over it