They came from voting in ancient Athens were people had a white ball and a black ball. You put one or the other into a jar, a black was a no vote, white was yes. It has never had anything to do with race. If it bothers you change the words for skin color instead then.

"whitelist-blacklist are names where you need to learn the meaning of them"

You could say this about every word. All language is based on past usage.