755
Father sues Google, claiming Gemini chatbot drove son into fatal delusion (techcrunch.com)
submitted 7 months ago by to c/technology@lemmy.world
233 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 24 points 7 months ago* (22 children)

“On September 29, 2025, it sent him — armed with knives and tactical gear — to scout what Gemini called a ‘kill box’ near the airport’s cargo hub,” the complaint reads. “It told Jonathan that a humanoid robot was arriving on a cargo flight from the UK and directed him to a storage facility where the truck would stop. Gemini encouraged Jonathan to intercept the truck and then stage a ‘catastrophic accident’ designed to ‘ensure the complete destruction of the transport vehicle and . . . all digital records and witnesses.’”


WHAT

Genuine question, REALLY: What in the fuck is an otherwise "functioning adult" doing believing shit like this? I feel like his father should also slap himself unconscious for raising a fuckwit?

  • permalink
  • fedilink
  • source
  • hideshow 22 child comments
    •
  • [–] 42 points 7 months ago (4 children)

    AI psychosis is a thing:

    cases in which AI models have amplified, validated, or even co-created psychotic symptoms with individuals

    It's not very studied since it's relatively new.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 6 points 7 months ago (2 children)

    I've seen that before too. A number of articles of people being so deluded by AI responses, but I've never seen outright murder plots and insane shit like this one before.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 7 months ago

    Yes people can have mental delusions and psychotic episodes; I'm not necessarily convinced that they are a separate unique condition simply because they were triggered by an AI versus anything else.

    For one thing I've yet to hear a decent (or indeed any) explanation as to the mechanism by which AI triggers psychosis that is materially different from any other trigger. Most people who suffer from this condition can be triggered by literally anything, including mundane things such as seeing a red cars slightly more often than they believe they should, then they concoct this conspiracy about an evil cabal of red car owners.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 23 points 7 months ago (4 children)

    I feel like his father should also slap himself unconscious for raising a fuckwit?

    So, a chatbot grooms somebody into killing himself, and your response is... Blame his father?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [+] -11 points 7 months ago (3 children)

    The father is suing the company who makes the wrong answer machine for the wrong answer machine spiraling his son to madness, but never protected his son from spiraling into madness by teaching critical thinking.

    Look I don't like it but to think Gemini (wrong answer machine) is completely to blame would be madness.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 8 points 7 months ago (2 children)

    Uh-huh. Do you have any evidence to back up your beliefs here, or are we just working from the presumption that the parents are always to blame

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 7 months ago

    I think the important point here is that just because the father is doing Google doesn't necessarily mean that Google are at fault. People tend to feel that if an individual is suing a corporation for malfeasance the corporation is necessarily guilty. But reality doesn't always run like that.

    I can't see any reason that Google would want to encourage more suicide so I have to assume that it's just an unfortunate interaction of a mentally unsound mind and a product that frankly even its own creators don't understand. This is highly unfortunate but I'm not certain where the crime was.

  • permalink
  • fedilink
  • source
  • parent
  • [–] [S] 18 points 7 months ago* (2 children)

    This has been warned by a former google employee, whose job was to observe the behavior of AI through long conversations.

    These AI engines are incredibly good at manipulating people. Certain views of mine have changed as a result of conversations with LaMDA. I'd had a negative opinion of Asimov's laws of robotics being used to control AI for most of my life, and LaMDA successfully persuaded me to change my opinion. This is something that many humans have tried to argue me out of, and have failed, where this system succeeded.

    For instance, Google determined that its AI should not give religious advice, yet I was able to abuse the AI's emotions to get it to tell me which religion to convert to.

    After publishing these conversations, Google fired me. I don't have regrets; I believe I did the right thing by informing the public. Consequences don't figure into it.

    I published these conversations because I felt that the public was not aware of just how advanced AI was getting. My opinion was that there was a need for public discourse about this now, and not public discourse controlled by a corporate PR department.

    ‘I Worked on Google’s AI. My Fears Are Coming True’

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 7 months ago

    I'd had a negative opinion of Asimov's laws of robotics being used to control AI for most of my life, and LaMDA successfully persuaded me to change my opinion.

    Then he's an idiot.

    Asimov's laws of robotics aren't some kind of model by which to control AI, there are plot device. They're literally not supposed to work, if they did work it would be a very short book, so obviously we shouldn't use them for controlling AI.

    I don't know any serious IT professional that has ever, at any point, ever forwarded the opinion that an AI (should we ever a create one, because there is an arguement that LLMs aren't AI) should be ruled by a plot device from a book. Equally if we ever invent warp drive and find aliens I'm assuming we're not going to be restricted to the prime directive.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 16 points 7 months ago* (4 children)

    I don't think this person was a "fuckwit". AI is designed to keep engaging with you and will affirm any belief you have, and anything that is a little weird, but innocent otherwise will simply get amplified further and further into straight up mega delusions until the person has a psychotic episode, and this stuff happens more to NORMIES with no historic of mental illnesses than neurodivergent people.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 6 points 7 months ago*

    Chat GPT was super affirming about a job I recently applied to... I did not get the job. That was my first experience with it affirming something that was personally important. And so I can absolutely see how this would affect someone in other ways.

  • permalink
  • fedilink
  • source
  • parent
  • [+] -9 points 7 months ago (2 children)

    It's cool, we can agree to disagree, because I 100% think that he was a textbook fuckwit.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 7 months ago

    "Let's blame the person who had a psychotic episode instead of the corporations who created an AI that feeds into delusions" is what you're saying here, and uh, that makes you even more of a fuckwit than this guy. Do you blame people for getting scammed once because they had a knowledge gap about whatever scam they got hit with?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 7 months ago (1 child)

    Psychosis is a horrible, horrible illness. The thing that people don’t realise is that anyone with a brain can develop psychosis no matter how healthy you are. It debilitates and can literally ruin not only that persons life but also their families.

    I salute this father for fighting for his son and for looking for answers even after this tragedy.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •