Being a colonizer is in some respects also a state of mind. You can choose to behave in such a way that is consistent with anti-settlerism, with respect for the land and for its indigenous and oppressed people. Being aware of your privilege is a very good first step. Feeling personally guilty, however, is not productive. You don't have to feel guilty so long as you choose to be part of the solution, to fight the colonial power structures, and stand in solidarity with those who don't have your privilege.

In fact i would go so far as to say that an obnoxious guilt culture is itself a sign of a self-indulgent and performative attitude. You perform guilt to yourself and others to feel better about yourself and virtue signal. But this self-flaggelation materially helps no one.

The most instructive case study on this is how German society has made guilt culture a matter of national pride in an extremely toxic way, patting themselves on the back about how morally superior they are because of how they "dealt" with their past, all while yet again supporting Nazis and backing a genocide.

But back to the original point: the solution to colonialism is not ethnic cleansing. It is to build a just society that restores power back to the colonized. Those who are incapable of accepting this new de-colonized state of affairs in which they no longer enjoy settler privilege will leave on their own.