So like a single ActivityPub instance that hosts all the data, but users can have a Pixelfed app, Lemmy app, etc. all connect to that one server and use it to give the experience they specifically provide.

That's a cool idea. I can see how that would work.