The way I understand it, no one should deal with those they don't want to, and that's best done bilaterally than having to deal with the choices of a central governance. And ActivityPub should go by this principle. Instances decide if they want relations with other instances. Users decide if they want relations with other users, communities, tags, and instances. If a given instance does something the user doesn't like, the user can pick another instance without inhering the choices of the previous, but still being able to access the environment and choose whom to deal with.

Also, in a process, whatever it may be, it's the point that more parts depend on that if it fails, the more damage happens. While AT Protocol's PDS system helps lowering stress on specific instances, the tendency is to have only a handful few, as "they already do all the job", so if a PDS node fails, more strain goes to the others, if the site owner chose or remembered to put back up nodes. Meanwhile, if an instance dies on ActivityPub, just a small part of the environment dies, but even posts from that dead instance may keep propagating. It's a situation of safety versus server optimization, I'd say.