In their defense, if everyone was perfect everything would be perfect.
But you're absolutely right. As long as humans are human there's no way to separate human nature from authority. And as a result, any system that doesn't strictly limit scope and authority is inherently bad.