If some chud tries to equate cracker with the n-word that's a fight worth having: persuadable people instinctively know we're right and you can educate people by explaining why. There is actually something to gain there, because the reason is basically structural racism, a topic most people do not adequately understand, much less confront.
What's silly is insisting that a term insulting one's skin color isn't racist at all. You get nothing out of that fight you don't already get out of "of course cracker isn't anywhere near as bad as the n-word," but now most people are thinking "I don't know, ripping on someone's skin color seems pretty racist to me."
Picking your battles is good, actually.