The biggest argument for the answer being “Yes” would seem to be their contributions to the Kessler Syndrome but I’m really not that up to speed on how severe or critical the situation currently is.
SpaceX wouldn't even exist without Musk and is deeply entwined with all his bullshit magical thinking or rather lying.
And as it stands right now their whole business model is deeply immoral, imo. Shooting shit into space, burning fucktons of fossil fuel, all to achieve things that could better be achieved with less effort. E.g. working on proper internet infrastructure in the USA instead of trying to solve the problem for rich people only.
But if you're asking if it would be a good idea to remove Musk from that company I'd still say yes.