Carefully, on a case-by-case basis; and the community.
It's not nearly as complicated as it seems on the surface - and you're trying to make any definition of "nazi" as complicated as possible, because you're wanting to delegitimize any rejection of nazis or nazi speech.
Remember how you said you don't care if people like you, you just want to push your topics on other people?
it diminishes its significance and undermines your credibility.
No one cares if the Nazis think they're "credible" or not. Each and every one of them will tell you they're not a nazi and they 'hate' nazis - while defending themselves and their nazi buddies from critique by insisting the label for their ideology is, for example, "cheapened" if applied to anyone who is not a card-carrying, armband-wearing, farcical exaggeration of stereotypical Nazis in full Reich dress regalia.
They always send the clean cut, quiet, polite one in first. And that guy puts a foot in the door, argues that their pals aren't really nazis, and that everyone in the room are the real baddies for judging those other guys unfairly - and tries to pry the door wider so their Nazi buddies can come in. Sure enough, every time, you let enough nazis in the room and the room is a nazi space now - so the whole gang of them don't have to pretend at being polite non-nazis anymore. The polite veneer, the deep care for "debate", and "respect for all viewpoints"? Those are all just tools, trying to whitewash and re-legitimize an ideology whose end goal is harming other people.
Notice how I'm casually referring to you like you're one of them? That's not some wokist over-use of the term. You're standing here defending them, you're trying to shove a foot in the door for them, laying down apologia for their views and their right to share them - you've spent like a week around the Fediverse arguing against any actions that have served to limit Nazis access to polite and adult spaces within the Fediverse as a whole. I don't care what you believe about yourself, or your views, or your ideology.
If you're going to stand with Nazis, if you're going to stand for them, consistently and repeatedly - don't get all offended and playact at being victimized when people assume that you are a member of the group you chose to stand with.