Sure, if you can replace it with something better. But if you can't do that, and you can just kill off something from your childhood that you enjoyed, that makes you a rather cold person, and then what's the point? If we haven't got compassion and love, what's left? Just coldness for those we hate? That's no way to live.
Do I regret that a book series written by a bigot got so big? Well, yeah. But we didn't know it at the time. I'm not going to apologise for something that caught my imagination 25 years ago.
Also, people aren't about just one issue. Boy George is gay, and he supports the genocide in Gaza. What does that mean? It just means one rich guy who used to make good music is a POS. Doesn't mean everyone who likes Karma Chameleon supports genocide. Though, I find myself not choosing to play it as much. Likewise, I don't read the Harry Potter books constantly. I just won't apologise for the past. I don't live there anymore.