709
Florida woman used Claude as a diary, then Anthropic reported an entry to police (www.techspot.com)
submitted 2 days ago by to c/technology@lemmy.world
218 comments fedilink hide all child comments
 

What just happened? Another incident has taken place that illustrates the need to be careful what you tell AI. A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to "shoot up" the Sheriff's office. After a human reviewer examined the statements, they were reported to police.

According to the arrest report, Carli Michelle Heller, of Bonita Springs, Florida, wrote on September 26 that she would attack the Sheriff's office. She later said that she uses Anthropic's chatbot like a "diary."

Claude's safety systems flagged the entry and it was escalated to a human reviewer. After deciding it was a credible threat, the reviewer reported it to law enforcement.

The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.

Deputies identified Heller and visited her home. She was detained without incident before an LCSO intelligence detective took over the investigation.

Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.

Anthropic isn't going to be taking any chances when it comes to anything it deems a potential threat. Last month, it was reported that OpenAI and Sam Altman are being sued by British Columbia over claims that the company could have prevented a mass shooting in the Canadian province.

The shooter, eighteen-year-old former pupil Jesse Van ⁠Rootselaar, had previously been flagged by OpenAI's safety team for her conversations about gun violence, but OpenAI never alerted police because the conversations did not meet the threshold for legal referral.

In June, Florida also sued OpenAI and Altman, alleging that ChatGPT had contributed to real-world harms, including the 2025 Florida State University shooting.

The latest incident is another reminder to think before you enter something into a chatbot that could get you into trouble. It's certainly not a private diary whose contents are for your eyes only.

Reports last month revealed that human contractors reviewing Microsoft Copilot's image editor can see users' prompts, uploaded photos and AI-generated edits. Documents show that some of those assignments contain sexual, disturbing or potentially illegal material, though the reviewers are not there to flag the content – only to assess whether the output is accurate.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 6 hours ago (2 children)

I don’t know if that was meant to reply to me, but I meant whether another human was involved in the training, not the threat. It’s definitely an imprudent thing to write, but I don’t know if a reasonable person would assume that another person would be able to read it. “Training” is a pretty nebulous concept, and the anthropic release doesn’t mention that humans are involved.

I’m not trying to say that this lady is making good decisions, but I probably wouldn’t vote that this was written so that another human might read it, if I were on the jury. Maybe my gut feeling’s off and everyone knows that training involves people (which I’m sure would come up during the trial), in which case I’d probably consider it a clear threat.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 hours ago (1 child)

    Look at this another way. People intend to do one thing and do something else entirely all the time. They're fallible. They make mistakes. The fact that it is a mistake doesn't mean there aren't consequences. The law is poorly written (the transmission part in particular should involve knowingly transmitting that threat where a human could be expected to read it) but she did pretty much exactly what the law says is illegal (assuming that the evidence against her is credible which would need to be investigated and proven).

    If you have a written diary and you happen to have the cleaner find it and read it and they report you to the proper authorities for it, your intent wasn't to have it read, but the actuality is that it happened.

    A threat you mumble to yourself out loud under your breath is heard without your knowledge? It's still a threat and can be reported as such. You can still be prosecuted for it.

    Legal Compliance. We may comply with governmental, court, and law enforcement requests or requirements relating to provision or use of the Services, or to information provided to or collected under our Terms. We reserve the right, at our sole discretion, to report information from or about you, including but not limited to Inputs, Outputs, or Actions to law enforcement.

    Anthropic’s Safeguards Team will implement detection and monitoring to enforce our Usage Policy, so please review this policy carefully before using our products or services. If we learn that you have violated our Usage Policy, we may throttle, suspend, or terminate your access to our products and services. We may also block or modify model outputs when inputs violate our Usage Policy.

    In addition to training our language models to refuse harmful requests, we also use dedicated Trust and Safety enforcement systems to detect, block, and take action on activity that might violate our Usage Policy. Clio supplements this work to help us understand where there might be opportunities to improve and strengthen these systems.

    We implement strict privacy access controls when it comes to who can use Clio to further enforce our policies because it may require review of individual accounts. Our Trust and Safety team is able to review topic clusters for areas that indicate likely violations of our Usage Policy.

    https://www.anthropic.com/legal/consumer-terms

    https://www.anthropic.com/legal/aup

    https://www.anthropic.com/research/clio

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 hour ago*

    Well yeah, but there are lots of situations where your intent with a threat matters. Saying “I’m gonna kill you!” to your brother when he throws a water balloon at you is rightfully not considered a legal threat. Saying “I finally bought a gun and I’m gonna kill you!” might fall into either category for me personally, depending on whether they had actually recently bought a gun.

    I’m really not trying to be difficult here and I do absolutely see how this could be ruled a legitimate threat, I just don’t think it’s 100%. I’m leaning towards 80% (assuming she didn’t actually recently get a gun), given that this is Florida.

    Regarding the diary, I was actually thinking of that exact scenario, with the stipulation that the diary is locked. I would be interested in whether that aspect of the law has been tested yet, but it’s probably not that common.

  • permalink
  • fedilink
  • source
  • parent