I also rush to take a label for me that I am this, this and this, and get myself limited to that specific identity.
It sounds like you know what the basic problem is: you find something, and take it as a part of your actual identity, letting it define you.
Instead of focusing on just one thing and labeling yourself with it, do many things - and have them be things that you do.
And sit down and think about what you're finding so attractive about these things, then work against that attraction. Is it because these groups offer a community? Then don't link yourself to just one community. Is it because it gives you a pre-set list of approved behaviors and opinions? Explore beyond the approved. Is it because you're not sure who you are and this makes you easier with yourself (something we all go through)? Go out and do a variety of things - do things by yourself, do them with others, go places and do things that you wouldn't normally.
Don't focus exclusively on one group or type of groups, look for a range of groups and activities. And if there's something interesting that you can't find a group for, go do it anyway, just because it interests you.
Good luck!