People Are Being Involuntarily Committed, Jailed After Spiraling Into "ChatGPT Psychosis" (futurism.com)

submitted 3 days ago* (last edited 3 days ago) by [email protected] to c/[email protected]

27 comments fedilink hide all child comments

Who are these people? This is ridiculous. :)

I guess with so many humans, there is bound to be a small number of people who have no ability to think for themselves and believe everything a chat bot is writing in their web browser.

People even have romantic relationships with these things.

I dont agree with the argument that chat gpt should "push back". They have an example in the article where the guy asked for tall bridges to jump from, and chat gpt listed them of course.

Are we expecting the llm to act like a psychologist, evaluating if the users state of mind is healthy before answering questions?

Very slippery slope if you ask me.

you are viewing a single comment's thread
view the rest of the comments

[-] [email protected] 10 points 3 days ago

I mean, having it not help people commit suicide would be a good starting point for AI safety.

[+] [email protected] -6 points 3 days ago* (last edited 3 days ago)

It will take another five seconds to find the same info using the web. Unless you also think we should censor the entire web and make it illegal to have any information about things that can hurt people, like knives, guns, stress, partners, cars....

People will not be stopped suiciding because a chat bot doesnt tell them the best way, unfortunately.

[-] [email protected] 8 points 3 days ago

This is also a problem for search engines.

A problem that while not solved has been somewhat mitigated by including suicide prevention resources at the top of search results.

This is a bare minimum AI can’t meet, and in conversation with AI vulnerable people can get more than just information, there are confirmed cases of the AI encouraging harmful behaviors up to and including suicide.

[-] [email protected] 3 points 3 days ago* (last edited 3 days ago)

It will take another five seconds to find the same info using the web.

good. every additional hurdle between a suicidal person and the actual act saves lives.

Unless you also think we should censor the entire web and make it illegal to have any information about things that can hurt people, like knives, guns, stress, partners, cars....

this isn’t a slippery slope. we can land on a reasonable middle ground.

People will not be stopped suiciding because a chat bot doesnt tell them the best way, unfortunately.

you don’t know that. maybe some will.

the general trend i get from your comment is you’re thinking in very black and white terms. the world doesn’t operate on all or nothing rules. there is always a balance between safety and practicality.

this post was submitted on 19 Jul 2025

82 points (81.5% liked)

Technology

73066 readers

2982 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago

MODERATORS

[email protected]