Eh, Nazi Germany basically attacked Poland to do settler colonialism (or at least that was the regime's justification). Main difference is that the Poles were more developed that e.g. the Native Americans.

IMO the error is assuming that it was likely to be successful and profitable, war/colonialism is generally not sound government policy.

It's a matter of distance really. Germany raiding France or Brazil isn't functionally different.