232
Stanford researchers find Mastodon has a massive child abuse material problem (www.theverge.com)
submitted 3 years ago by to c/technology@beehaw.org
123 comments fedilink hide all child comments
 

Mastodon, an alternative social network to Twitter, has a serious problem with child sexual abuse material according to researchers from Stanford University. In just two days, researchers found over 100 instances of known CSAM across over 325,000 posts on Mastodon. The researchers found hundreds of posts containing CSAM related hashtags and links pointing to CSAM trading and grooming of minors. One Mastodon server was even taken down for a period of time due to CSAM being posted. The researchers suggest that decentralized networks like Mastodon need to implement more robust moderation tools and reporting mechanisms to address the prevalence of CSAM.

you are viewing a single comment's thread
view the rest of the comments
[–] 72 points 3 years ago (17 children)

Yeah I recall that the Japanese instances have a big problem with that shit. As for the rest of us, Facebook actually open sourced some efficient hashing algorithms for use for dealing with CSAM; Fediverse platforms could implement these, which would just leave the issue of getting an image hash database to check against. All the big platforms could probably chip in to get access to one of those private databases and then release a public service for use with the ecosystem.

  • permalink
  • fedilink
  • source
  • hideshow 17 child comments
    •
  • [–] 14 points 3 years ago (13 children)

    That'd be useless though, because first, it'd probably opt-in via configuration settings and even if it wasn't, people would just fork and modify the code base or simply switch to another ActivityPub implementation.

    We're not gonna fix society using tech unless we're all hooked up to some all knowing AI under government control.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 13 child comments
    •
  • [–] 19 points 3 years ago (3 children)

    That’d be useless though, because first, it’d probably opt-in via configuration settings and even if it wasn’t, people would just fork and modify the code base or simply switch to another ActivityPub implementation.

    No it wouldn't, because it'd still be significantly easier for instances to deal with CSAM content with this functionality built into the platforms. And I highly doubt there's going to be a mass migration from any Fediverse platform that implements such a feature (though honestly I'd be down to defederate with any instance that takes serious issue with this).

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 3 years ago (2 children)

    And the instances who want to engage with that material would all opt for the fork and be done with it. That's all I meant.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 13 points 3 years ago (8 children)

    That's not the point. Yes, child porn sites can host child porn. Other sites/instances can't stop that. But what other instances can stop, is redistributing said child porn. And for that purpose, such technology would be useful.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 5 points 3 years ago (7 children)

    researchers found 112 instances of known CSAM across 325,000 posts on the platform

    So you're willing to vacuum up the hashes of every image file uploaded on thousands of decentralized systems into a centralized systems (that is out of "our" control and coupled with direct access for law enforcement and corporations) to prevent the distribution of 0.034% of files that are CSAM and that could just as well be reported and deleted by admins and moderators? Remember how Snowden warned us about metadata?

    If you think that's a wise tradeoff, I guess, go ahead. But then I'd have to question the entire goal of being decentralized in the first place. If it's all about "a billionare can't wreak havok upon my social network", then yeah, I guess decentralization helps a bit but even that remains to be seen.

    But if you're actually willing to do that, you'd probably also be in favor of having government backdoors into chat encryption (and thus rendering the entire concept moot, because you can't have backdoors that cannot be discovered by other nefarious actors) and even more censorship-resistant systems like Tor because evil people use it to exchange CSAM anonymously as well?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 5 points 3 years ago (1 child)

    If you read the article, there's actually more. The problem also isn't just that they post the material directly onto Mastodon, they also use the platform to network.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 3 years ago*

    More... or less, given that US-centric CSAM detectors mark AI, CG and drawings at the same level as IRL images.

    Preventing the networking, is called defederation, that's already there.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 3 years ago* (4 children)

    I don't know how you get the impression that this increases censorship.

    Instance admin already manually block content. And they are already able to do that to any extend they wish to do.

    This tool would simply automate that process.

    Admins would not gain or lose any ability to block content. Identifying child porn would simply be easier.

    (Imagine an admin going to their database and doing a CTRL+F with the term "child porn", and then going through the posts to find offending ones. But instead of CTRL+F it's an AI.)

    (For some reason I don't get a notification when you answer my comment. Is that a known issue? Did you block me or something?)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 3 years ago*

    I'm referring to the CSAM scanning systems that are outside of the control of almost anyone except governments, three letter agencies, other law enforcement and parts of the private sector.

    These systems must be fed every hash of every file submitted to as many instances as possible to be efficient with close to no oversight or public scrutiny.

    Pass.

    Edit: I'm not blocking you but I noticed intermittent connectivity issues on lemmy.ml today, possibly around the time where I replied.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 3 years ago (2 children)

    I don't know how you get the impression that this increases censorship.

    This tool would simply automate that process.

    Well... precisely?

    Censorship is any removal of material considered "undesirable", whether you agree with why it is considered "undesirable" or not.

    If you want more censorship of "material that you personally consider undesirable", then just say so, don't hide behind some disingenuous "but it isn't censorship". Then we can discuss the merits of that classification, and of the means proposed to achieve such censorship.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 years ago (1 child)

    You seem to be missing my point.

    This tool would not increase censorship.

    Admins are already able to implement all censorship they want.

    Admins are already able to block left-wing opinions, right-wing opinions, child porn, normal porn.

    And that already happens.

    Lots of instances (like feddit.de) block pornographic content.

    Lots of instances (like lemmy.blahaj.zone) block right-wing content.

    It is already possible, and it is already happening.

    An AI which can detect CSAM (and potentially other content) won't change that. It will simply make the admins' job easier.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 years ago

    I think you're missing the opposite point.

    An AI trained on a given instance's admin decisions, would increase the same censorship the admins already apply. We can agree on that.

    An AI trained by a third-party on unknown data (and actually illegal to be known) which can detect "CSAM (and potentially other content)", would increase censorship of both CSAM... and of "potentially other content" out of the control, preferences or knowledge of the instance admins.

    Using an external service to submit ALL content for an AI trained by a third-party to make a decision, not only allows the external service to collect ALL the content (not just the censored one), but also to change the decision parameters without previous notice, or any kind of oversight, and apply it to ALL content.

    The problem is a difference between:

    • instance modlog -> instance content filtered by instance AI -> makes similar decisions as instance admins
    • [illegal to know dataset] -> third-party captures all content, feeds to undisclosed AI -> makes unknown decisions in the name of removing CSAM

    One is an AI that can make mistakes, but mostly follows whatever an admin would do. The other, is a 100% surveillance state nightmare in the name of filtering 0.03% of content.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 11 points 3 years ago (2 children)

    Facebook actually being good for once? Unheard of

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •