A mother and her 14-year-old daughter are advocating for better protections for victims after AI-generated nude images of the teen and other female classmates were circulated at a high school in New Jersey.

Meanwhile, on the other side of the country, officials are investigating an incident involving a teenage boy who allegedly used artificial intelligence to create and distribute similar images of other students – also teen girls - that attend a high school in suburban Seattle, Washington.

The disturbing cases have put a spotlight yet again on explicit AI-generated material that overwhelmingly harms women and children and is booming online at an unprecedented rate. According to an analysis by independent researcher Genevieve Oh that was shared with The Associated Press, more than 143,000 new deepfake videos were posted online this year, which surpasses every other year combined.

you are viewing a single comment's thread
view the rest of the comments
[–] 112 points 2 years ago (21 children)

I don't know what a reasonable"protection" looks like here: the only thing foresee is 14 year old boys getting felonies, but no one being protected.

  • source
  • hideshow 21 child comments
  • [–] 57 points 2 years ago (14 children)

    Right, there are plenty of reactive measures available but the only proactive measures are either restricting availability of the source photos used or restricting use of the deep fake tools used. Everything beyond that is trying to put the genie back in the bottle.

  • source
  • parent
  • hideshow 14 child comments
  • [–] 52 points 2 years ago (8 children)

    At some point, communities and social circles need to be able to moderate themselves.

    Disseminating nudes of peers should be grounds for ostracizing, but it really depends on the quality of people around you.

  • source
  • parent
  • hideshow 8 child comments
  • [+] 19 points 2 years ago* (last edited 1 year ago) (7 children)
  • [–] -2 points 2 years ago (6 children)

    Ostracization doesn’t work. Ever. Period. If they’re bad enough, banishment works. Ostracization is just literally ignoring the problem.

    That's just wrong. Unless you're hanging around shitty people, ignoring the bad ones by definition works.

  • source
  • parent
  • hideshow 6 child comments
  • [+] 2 points 2 years ago* (last edited 1 year ago) (4 children)
  • [–] 0 points 2 years ago (3 children)

    Yeah, it does. It's literally how most friend groups keep undesirables from hanging out with them.

  • source
  • parent
  • hideshow 3 child comments
  • [+] 4 points 2 years ago* (last edited 1 year ago) (2 children)
  • [–] 2 points 2 years ago (1 child)

    I know. I've also said that "if the microcosm is so shitty that it doesn’t ostracize people who disseminate nudes, then the people in it deserve to suffer until they improve [or leave.]"

    Not everyone is worth hanging around. I know it's difficult to be alone and most people would rather have shitty 'friends' than be alone, but that doesn't mean they won't suffer from being around shitty people.

    I think the answer is to take the power away from shitbags by avoiding them. The more people who do this, the less influence shitbags have.

  • source
  • parent
  • hideshow 1 child comment
  • [+] -13 points 2 years ago (1 child)

    Are we seriously going to try and use someone's photos for dumb shit like this? Cone on, people just want something to wank to or someone to call over to have sex with, who the hell would actually do this?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 15 points 2 years ago (3 children)

    Even if you don't want to consider it CSAM, it is, at the very least, sexual harassment. The kids making and circulating these pictures and videos should be facing consequences. And the fear of consequences does offer some degree of protection at least.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 2 years ago

    It looks like pretty severe sexual harassment at best. Unfortunately the people I think are most likely to do it are teenagers with poor self control who don't realize the severity.

    I think if schools can implement appropriate restorative responses and education on the harm done that could be much more effective than decaigan punishments after the fact.

  • source
  • parent
  • [–] 0 points 2 years ago (1 child)

    Should a teenager face consequences for drawing a picture of their classmate naked? What if they do it well? How is this at all different?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 2 years ago

    If they distribute the drawing, yes. And the difference is that a drawing is immediately recognisable as a drawing, but an AI generated image or video isn't necessarily easily recognisable as not being real, so the social consequences for the person depicted can be much worse.

  • source
  • parent