The space here is limited... The immigrants are directly competing with me for the in-city apartments.

More desirable locations will be too expensive for many, or most, people to afford. As local economies change, and different locations become desirable, people will be priced out and forced to move. Good city planning decisions can slow this down to allow people to adapt, but trying to freeze things in place is futile.

It's not really possible to set up city planning regulations so the population stays exactly the same. If a city were successful in making itself an undesirable place to live so that no one new would move there, it would probably start losing its population, which (like growth) forces its own hard planning decisions.