I believe the battles etc all balanced for a party of 4. For games like this, creating a challenging battle for a larger party typically elongates the combat duration waiting for more enemies to make their moves or enemies becoming damage sponges. This may make the game's combat a bit too exhausting/boring for the general audience.
There maybe overhaul mods that could allow for larger parties without impacting or even increasing difficulty, e.g. long war for xcom EU and xcom 2.
Personally I would prefer a bigger party since I want to use more classes in my party, or troll around with a 6 paladin crusader party for the LOLs.