I've also travelled and that's why I want borders, at least for now.
We should definitely be working to raise people in the poorer parts of the world out of poverty. I've been to some developing countries, and life there is extremely hard. That's why so many of them want to move to a rich country where they can live a better life.
Yeah, as a rich westerner, borders are an annoyance. It's great going around Europe and going from the Netherlands to France without needing a passport or anything.
Having said that, in the Netherlands and France, and most other western countries, there is a great social safety net paid for by taxes that makes sure that everybody living in those countries can live a decent life. But, even Europe was put under stress by the fact that some countries in the EU have a GDP per capita that's half what it is in other places.
A fair number of people migrated to richer countries within the EU for a better life. Most found a job and worked. They contributed more to the countries than they received in benefits. Even then, that was hard on those richer countries because it displaced some of the lower-wage workers, or depressed their wages. Despite this, the EU has mostly held together, other than Brexit, and the majority of the UK now regrets Brexit.
The important thing to remember is that the EU vets members before allowing them in, and was slow to allow the poorer Eastern European countries to join.
If the world eliminated borders tomorrow, the richer western countries would be swamped by poorer people from developing countries seeking a better life. There's no way that the systems in those countries could cope.
The rich might love it. Suddenly there would be hundreds or thousands of people competing for the lowest paying jobs, allowing the bosses to offer the lowest possible wages, the worst possible working conditions, etc. The ultra-rich business owners could get rid of unions and get local workers for next to nothing.
For the slightly less rich, the professional class, it wouldn't be too bad. Their jobs are protected. You need to pass the bar to be a lawyer. You need to do a residency to become a doctor. Their professional associations would continue to do the work necessary so that say a doctor in Nigeria didn't have their credentials recognized in the west.
The less wealthy you are, the less protected you would be. For someone whose job doesn't require any kind of diploma, certificate or degree, there would be no protection. They'd be competing with every unskilled person who came into the country seeking opportunities.
Even if you personally think that chaos is a small price to pay for a more fair world, a lot of people don't. The rise of far-right and fascist movements in the west is a direct result of increasing immigration. If borders were eliminated, not only would there be a lot more far-right voters, there would probably be violent clashes, even mass slaughters.
A lot of people hate globalization, but it has resulted in raising a lot of people in developing countries out of extreme poverty. It has done that mainly by taking wealth and power away from the poorest people in richer countries by shipping their jobs overseas. That part sucks. But, at least a lot of people in developing countries are no longer in danger of dying if they have a bad few weeks.
Programs like USAID were a better way of doing some of the same things. These were often subsidies for US farmers whose produce was then just donated or sold extremely cheaply to developing countries. There were of course problems with this system too. Often the "farmers" receiving the subsidies were rich corporations. But, it did raise the floor for some of the worst-off people in the world.
If we can continue to find ways to help people in developing countries, we should do that. Hopefully we can find a better way to do it than we've done in the past. But, even a less-than-ideal solution does help. Eventually, we may be able to reach a point at which we can remove borders without resulting in a mass migration that overwhelms the systems of the richer countries.