One can have segregation and cheap colored abused labor without slavery, like most of the British empire after it went out of fashion. And one can rob, abuse and genocide natives with a formal truce in place.

Those pesky colonials just didn't understand nuance.