Because people fear having their culture and race replaced by immigrants. Even if they're not overtly racist, few people wish to become a minority in "their own country."
The US is famously a melting pot, and yet we still have a bunch of descendants of white immigrants from Europe who fear that South Americans will take over; that Mexican culture will replace good old-fashioned hodge-podge Western European culture. That their language will become less dominant. That they'll find themselves strangers in their own country.
It's usually an indistinct fear. It seems obvious from the verbiage in the dog-whistles, but white European immigrant descendants don't want to become second-class.
Now, if we treated our own minorities well, they wouldn't be so afraid. They wouldn't be afraid that they'd be the ones with Hispanic cops kneeling on their necks; or that Hispanic immigrants would be living in giant homes and they'd themselves be the ones having to eak out a living as seasonal workers.
I think it's not despicable to want to preserve your cultural heritage, your cultural language, and to have your country legislated with the values you grew up with; but people react poorly when they think it's happening.
What I most despise in the Republicans in the US is that they're advocating for preserving cultural values that never existed broadly in the US. The closest subculture to what they're pushing is a return to the Confederate South: religion, and white supremacy. The Confederates got their asses handed to them, but the racist fuckers never gave up their values, most most Americans are blind to what their real agenda is. And they've been good insurgents, cleverly taking advantage of weak areas in our democracy to return power to a minority: themselves. It's been said and it's true: if America was a true democracy and we selected leaders by popular vote, no Republican under their current platform would ever be president again.
Anyway, getting back to your question: immigrants bring their own culture with them, and very few completely abandon it and adopt the culture and language of their new country. This dilutes the host country's native culture, and people are afraid of that. In the US, it's the highest form of hypocrisy, because our native culture displaced the indigenous culture, and now we're afraid of someone else doing the same to us.