Technology

59559 readers

3339 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

[email protected]

134

Child safety org launches AI model trained on real child sex abuse images (arstechnica.com)

submitted 1 day ago by [email protected] to c/[email protected]

85 comments fedilink hide all child comments

Today, a prominent child safety organization, Thorn, in partnership with a leading cloud-based AI solutions provider, Hive, announced the release of an AI model designed to flag unknown CSAM at upload. It's the earliest AI technology striving to expose unreported CSAM at scale.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 9 points 7 hours ago* (last edited 7 hours ago) (2 children)

Jesus Christ. If someone ever got their hands on this model they could use it to generate new material. The grossest possible AI model to date

[–] [email protected] 11 points 5 hours ago (1 children)

No. This is an inference model, not a generative model. You generally cannot train a model for both, unless you do it on purpose, and they certainly did not (especially since inference models are way easier to train than generative models).

[–] [email protected] 3 points 5 hours ago* (last edited 5 hours ago) (1 children)

A generative model uses the classifier as part of its training. If you generate a picture of pure random noise, then iteratively pick random noise that the classifier says "looks" more like csam, then you can effectively generate images that the classifier says it's 100% certain is csam. Whether or not that looks anything like what a human would consider to be csam depends on other factors but it remains a possibility.

[–] [email protected] 2 points 1 hour ago

You are describing the way deepdream works, not the way modern Diffusion models work. It's the difference between psychedelic dog faces and a highly adherent generative image of a German Sheppard.

I can't imagine you're going to get anything out of this model that actually looks like CSAM, unless there's some sort of breakthrough in using these models for previously unrealized generative purposes.

[–] [email protected] 1 points 7 hours ago

I thought being able to do that was already a thing. This is designed to do the opposite.

I know, I know.. bad actors and such.