As a former anarchist turned ML this was actually what made me begin to question anarchism. There's a lot of focus on how centralized and organized oppression, but in history there's plenty of examples of decentralized disorganized oppressive violence, especially when it comes to settler colonialism. Like in the West Bank a lot of these zionist gangs aren't particularly well organized, they're just roving bands of teenaged thugs, some so disorganized and violent even the Israeli state (at least up till recently) didn't want to be seen (directly) supporting them.

I think we need to recognize that some populations are so reactionary that letting them organize autonomously isn't going to lead to anything emancipatory. Like if you somehow turned the segregation south into a confederation of anarchist communes I think a lot, if not most, of the majority white ones would be like "okay we still hate black people so we're gonna directly democraticly vote to enforce segregated locally". If Washington had been taken over by Bolsheviks and they decided to roll the American Red Army in to force desegregation in the south would that be "authoritarian"?