That is one of the fundamental flaws of machine learning like this, the way they are trained means they end up always trying to agree with the user, because not doing so is taken as being a "wrong" answer. That is why they hallucinate answers too - because "I don't know" is not an acceptable answer, but generating something plausible that the user takes as truth works.
You then have to manually try to reign them in and prevent them from talking about things you don't want them to, but they are trivially easy to fool. IIRC, in one of these suicide cases the LLM did refuse to talk about suicide, until the user told it it was all just for a fictional story. And you can't really "fix" that without completely banning it from talking about those things in every single occasion, because someone will find a way around it eventually.

And yeah, they don't care, because they are essentially just predictive text algorithms turned up to 11. Chatbots like ChatGPT and other LLMs are an excellent application of both meanings of the word "Artificial Intelligence" - they emulate human intelligence by faking being intelligent, when they in reality are not.

And that is because they get their vast, innumerable sums of digital money from world governments! Human people are allowing an advertising and surveillance tool to Wormtongue its way into their heads and their lives because it breathlessly encourages and agrees with everything they think.

I just don't believe that our perceptions and ability to handle enthusiastic sycophantic agreement is evolved enough yet to combat something like this. I could see it being intoxicating to anyone for everything they say to be agreed with, confirmed, and called genius. I don't necessarily blame the people falling for it (though I do think adults who fall for it are a bit sad and need to grow up a bit), but it's definitely going to be massively convenient for governments to have their citizens just voice everything they're thinking.

Sort of like Minority Report but everybody says their own future crimes outright to a little robot butler instead.

You must have used ChatGPT a lot to say this, because that's completely false. There are safeguards for both things