This. Whenever people use "if we don't eat meat we need this much less land" I'm immediately thinking if we don't need to plant all that grass and other things then people would just make more houses on those land not grow a forest.

Yes, but the development on the right is going to discover the colony of cannibalistic cave dwellers much quicker, as the high density makes it more difficult to hunt unseen.