They enforce their authority on others, by force when necessary they are "authoritarian". Being accountable to their supporters in their areas doesn't change that. Until their is a global overthrow of class society by necessity their will be power structures where one class has authority and another doesn't. All governments/movements/classes that have been or are, are "authoritarian".

This renders authoritarian a useless term for analysis and as such has relegated it to being used to paint groups/movements/governments as evil or immoral should they stand outside liberal sensibilities.

If you wield force to protect your class interests (which is inevitable while there is a capitalist and a working class), then there is nothing that prevents the accusation of "authoritarian" against you, regardless of how "decentralized" you create your structures.

Not only will capitalists call you authoritarian for fighting to defend your interests, its extremely common for anarchists to accuse even other anarchist groups of being more "authoritarian" than their own. Its a meaningless term in a world with irreconcilable classes and class conflict.