232
Father sues Google, claiming Gemini chatbot drove son into fatal delusion (techcrunch.com)
submitted 6 months ago by to c/fuck_ai@lemmy.world
19 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 17 points 6 months ago (7 children)

This is complete insanity. They clearly have no idea how to implement effective safeguards.

  • permalink
  • fedilink
  • source
  • hideshow 7 child comments
    •
  • [–] 19 points 6 months ago (3 children)

    Because it's not possible.

    LLMs are just machines that generate text. The text they generate is text that is statistically likely to appear after the existing text. You can do "prompt engineering" all you want, but that will never work. All prompt engineering does is change the words that come earlier in the context window. If the system calculates that the most likely words to come next are "you should kill yourself" then that's what it's going to spit out.

    You could try putting a filter in there to prevent it from outputting specific words or specific phrases. But, language is incredibly malleable. The LLM could spit out thousands of different ways of saying "kill yourself", and you can't block them all. If you want to try to prevent it from expressing the concept of killing one's self, you need something that can "comprehend" text... which at this point is just basically another version of the same kind of AI that generates the text, so that's not going to work.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 6 months ago* (2 children)

    I didn't feel like writing a long comment, but yes, good explanation! We really need to reign in these companies because their products are fundamentally untrustworthy.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 6 months ago (1 child)

    I think you’re missing the point entirely… it’s a fucking text generator. If you think it’s your ai wife and you commit suicide to be with her your a fucking nut job and the ai was just predicting fucking text

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 5 months ago

    Text has power. People have done worse because of what they read on 4chan.

    And sure, yeah, a well-adjusted person isn't going to end up thinking chatgpt is their wife, but it's not like anybody has access to mental healthcare around here.

  • permalink
  • fedilink
  • source
  • parent
  • [–] -1 points 6 months ago (2 children)

    Well, I actually noticed something recently. I pushed through a days worth of solo driving about a year ago. Repeated the same haul just this weekend. Both times I used Geminis voice chat for traffic updates, nearest points of interest and general chat. Far far different than the last time. The safe guards felt very in place now. Maps was cleaner integrated so it was a good copilot there, but general chat really went down hill.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •