Growing up and even now, I have a problem of going into extremes. In my early teens, I was so obsessed with religion that it started to become a problematic part for those around me. Now, I am kinda "anti-theist" (in secret though, I wanna be safe). I used to be very homophobic. When I discovered that I fall in love with a guy, I have became so scared. Now, I am comfortable with my bisexuality though but will be in rage if the word "gay" is used as a joke.
I also rush to take a label for me that I am this, this and this, and get myself limited to that specific identity.
Although I am now on my way to develop the acceptance of the fact that I can be wrong, I still feel that I have not gotten rid of my extremism.
How to be a person who does not rush to take labels?
It bears repeating that religious zealots and anti-theists are two sides of the same coin. To the point where anti-theists who were religious zealots is common, like you describe. Both offer certainty, superiority, and are obsessed with out-groups and enemies. You may want to reflect if, and perhaps why those things appeal to you.