451
ChatGPT convinced an Alabama woman to end her life to fulfill a divine prophecy, lawsuit alleges (www.al.com)
submitted 2 months ago by to c/technology@lemmy.world
137 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 1 month ago (4 children)

You’re underselling what companies can do. An LLM can’t verify reality the way people imagine, but it can detect patterns that correlate with crisis well enough to escalate, refuse harmful advice, or encourage real help. Seatbelts don’t prevent every crash either: they still reduce harm.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 1 month ago (3 children)

    I did say in my comment I'm for some guardrails of sorts as you suggested. Just not sure if a company is going to put time and resources into that - probably not voluntarily at least. Or if it'll compromise how "good" their product is. I'm starting to think AI might be more of a mirage at this point that's going to implode but I also kind of don't want the economy to explode at the same time?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 1 month ago (2 children)

    I think the bubble will not burst but deflate long before the tech disappears: Plenty of companies will burn cash and investors will get wrecked, but the genuinely useful products will stick around. Guardrails will improve too, because trust eventually becomes a competitive advantage.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 1 month ago (1 child)

    Guardrails will improve too, because trust eventually becomes a competitive advantage.

    Thing is, as an LLM user you don't want guardrails usually. Someone else wants it to have guardrails so you can't get all the information you want.

    If you're reverse engineering something, you don't want the guardrails to say "hey this is copyrighted, I can't do this". I'm assuming terrorists don't want the guardrails to stop the LLM from giving them info on chemical weapons. And if you're religious, you probably don't want guardrails to stop the LLM from talking about religion with you

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 0 points 1 month ago

    You’re describing different guardrails: Most users don’t want arbitrary censorship, but almost everyone benefits from guardrails that reduce clearly harmful or illegal outputs. The real debate is not whether they exist, it is who decides where the line is drawn.

  • permalink
  • fedilink
  • source
  • parent