"Have you considered doing a flip as you leap off the building? That way your death is super memorable and cool, even if your life wasn't."

-Crisis hotline LLM, probably.

Less horrifying conceptually, but in Canada a major airline tried to replace their support services with a chatbot. The chatbot then invented discounts that didn't actually exist, and the courts ruled that the airline had to honour them. The chatbot was, for all intents and purposes, no more or less official a source of data than any other information they put out, such as their website and other documentation.

They know the tech is not good enough, they just dont care and want to maximise profit.

I was talking to a friend recently about this. They studied medieval English and aren't especially techy, besides being a Millennial with techy friends; I said that merely knowing and using the term LLM correctly puts their AI knowledge above the vast majority of people (including a decent chunk of people trying to make a quick buck off of AI hype)

I agree. The only application that is fine for this in my opinion is using it solely for entertainment, as a toy.

The problem is of course that everyone and their mothers are pouring billions into what clearly should only be used as a toy, expecting it to perform miracles it currently can not and might never be able to pull off.