Eh, it's a big galaxy. You've also got the Tollan, the Nox, and the Aschen running around in their respective parts of the galaxy.

There were, what, roughly 210 episodes of the show? So we're picking about 200 random places out of 100 billion stars. There's still a lot of room for entire Federations worth of stuff in other parts of the galaxy (sorry to cross fandoms for that comparison, lol).