The audience is not the problem. Meta's mere presence on the network will be. We are now at a critical point in the struggle to survive as a network, and it's not looking good.
If we continue like today, the network effect (Google it) would eventually lead to ActivityPub being the de facto too-big-to-fail standard in all of the web. We aren't there yet, though. Meta knows this too and doesn't want it to happen, because extracting value from a diverse network is way harder than from a centralized user base. The fact that they even want to federate in the first place (shouldn't be in their interest!) rings alarm bells.