No, it's very much justified in my books. Who am I to tell you what to find offensive or not? It's very much subjective at the end of the day. As a queer person myself, I do agree with you that language as a whole should be striving to be as inclusive as possible and using words like "gay" as an insult definitely doesn't help. Wasn't insinuating that I find their using it that way ok.
I see people using the r-word as an insult on here quite often, for example, and am always happy to point it out and pick a fight. So no worries. I get you.