Agreed. Some level of xenophobia is expected though. That’s especially true if you were to think that the other tribes phenotypes are reminiscent of disease or sickness. I’m pretty sure that what you’re pointing out about wars starting with land isn’t so much because humans want to protect their land but because settling in one spot makes one more xenophobic by nature. Your breeding group is now smaller so phenotypes become distinct. Which means that encountering outsiders or wanderers would increase that xenophobic response. Prior to that you’d have a more or less homogenous group within anywhere you could reach which means that xenophobia makes far less sense as you cannot always identify “tribe”.

To sum up what I’m saying: over time as complexity began, cooperation between phenotypes lessened and competition between distinct phenotypes emerged. This could be what drives a lot of human xenophobic behavior, especially in the modern day. As we developed language this got even worse as xenophobia now includes a language barrier. Meaning that outsiders marked themselves as such even if they held the same phenotypes. This led to a lot of tribalism and history has not evolved from there.

(Just a layman but I did some reading and thinking)