You can both be vehemently anti-immigration and pro minority rights at the same time. It's mental gymnastics at its core, I agree, but it exists.
Very few people hold a cohesive worldview that holds up against total scrutiny, and the one's who do probably spend too much time inside their heads convinced of their own sanctity.