It really doesn't matter where they are from, ancestry doesn't excuse genocide. You could apply your friend's logic to defending colonialism in Africa, because on a long enough timeline, all humans originated from there, so Europe really had a legitimate claim to those colonies.