IMO, those are very different situations, with slavery their freedom was taken. Most of the migrants (not refugees) are free, they are simply in search for a better life. Which is understandable, don't get me wrong. But the picture that is painted is too bright, people hop on a boat to "paradise" and are promised a life of riches. The reality is, this is not the situation they will find themselves in. Even if we were to allow them into the country, they would still live poor lives. Sure some will succeed, but most of them don't have the foundational knowledge/skills/command of the language of the country to get a job and find a place to live.

Meanwhile, looking at my own country The Netherlands, we see that natives are also struggling for housing. My generation is basically fucked, young families cannot buy nor rent a home to start their family in. This will impact The Netherlands for years/decades to come, it will be reflected in birth rate and future workforce. Allowing other people in the country now would be catastrophic.

On the other hand, recently there was an article about an approach by French farmers which I like. They sourced Moroccan workers, that could work in France for a couple months (IIRC max stay was 3 months), during this time, the workers would receive fair pay. After the work permit ends, the workers have to go back to Morocco. Then they are only allowed to go back to work in France after a period of time, with a new workers permit.

The key benefit is that workers bring the money back home, benefitting Morocco financially, and France with workforce. Most of this work by the farmers is seasonal anyway.