I think local culture also has a hand in it. I'm in the Pacific Northwest, and we've got a serious hiking culture up here that is both obsessive and gatekeep-y.
I have been on hikes with rockhounds, and that's been great, but they don't call themselves hikers.