Yes, it WAS fucked up. That's how it was back in those days.
There was all kinds of stupid racism back in those days, and "science" (but actually pseudoscience) was used as an excuse for it, like Calvin Candie in Django Unchained going on about skull dimples.
Well, I say "back in those days" but really it's alive and well in the present, too. Have you ever read what modern racists believe? It's still nuts.
"Free"?