And will we get that technology to keep the Fediverse and free platforms safe? Probably not. All the predecessors have been kept away for sole use of the big players, despite populism always claiming we need to introduce total surveillance to keep the children safe...
this post was submitted on 21 Nov 2024
134 points (97.9% liked)
Technology
59559 readers
3339 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
I was going to say... Sure would be nice to have this feature in all the open source AI image generator tools but you're absolutely right 😩
Yeah, unless someone publishes even a set of hashes of known bad content for the general public... I kind of doubt the true intentions are preventing CSAM to the benefit of everyone.
You never know. @[email protected] / @npub1q3sle0kvfsehgsuexttt3ugjd8xdklxfwwkh559wxckmzddywnws6cd26p@momostr.pink tried getting Microsoft to provide their CSAM blocking filtering to the Fediverse in the past without success, but this is a different group.
If everyone has access to the model it becomes much easier to find obfuscation methods and validate them. It becomes an uphill battle. It's unfortunate but it's an inherent limitation of most safeguards.
You're probably right. I'm not sure if it's a good idea to walk close to the edge with things like this, though. Every update to the detection model could change things and get them in jail.... So I certainly wouldn't play a cat and mouse game with something that has several years of jailtime attached... But then I don't really know the thought process of the average pedo. And AI image detection comes with problems anyways. In the article they say it detected 6 million pictures already. While keeping quiet about the rate of false positives. We know people have gotten in serious trouble for (false) claims. And I also wouldn't want to be the Fediverse admin who has to go through thousands of flagged pictures and look at them and decide which is which. With consequences attached... Maybe a database of hashes would be the only option. That doesn't detect new pictures, but at the same time it comes without flase positives and you can't draw conclusions from hash values.
load more comments
(5 replies)
This seems like a potential actual good use of AI. Can't have been much fun to train it though.
And is there any risk of people turning these kinds of models around and using them to generate images?
If AI was reliable, maybe. MAYBE. But guess what? It turns out that “advanced autocomplete” does a shitty job of most things, and I bet false positives will be numerous.
This is not that kind of AI.
It's possible to have a good AI system, but it takes millions of dollars and several thousand manhours to do, and most companies won't put in the effort.
But, there should always be a human in the loop.
"detect new or previously unreported CSAM and child sexual exploitation behavior (CSE), generating a risk score to make human decisions easier and faster."
False positives don't matter if they stick to the stated intended purpose of making it easier to detect CSAM manually.
if they stick to the stated intended purpose
They never do.
The problem is that they won't.
Yes, AI tools, in the hands of skilled people, can be very helpful.
But "AI" in capitalism doesn't mean "more effective workers", it means "fewer workers." The issue isn't technological so much as cultural. You fundamentally cannot convince an MBA not to try to automate away jobs.
(It's not even a money thing; it's about getting rid of all those pesky "workers rights" that workers like to bring with us)
Here's the thing. This technology is unequivocally one of the things AI would be very useful for. It can potentially do a lot of good. Yes, MBAs could screw it up like they screw anything else up in society. That doesn't mean we shouldn't be happy that we've created this new tech.
And is there any risk of people turning these kinds of models around and using them to generate images?
There isn't really much fundamental difference between an image detector and an image generator. The way image generators like stable diffusion work is essentially by generating a starting image that's nothing but random static and telling the generator "find the cat that's hidden in this noise."
It'll probably take a bit of work to rig this child porn detector up to generate images, but I could definitely imagine it happening. It's going to make an already complicated philosophical debate even more complicated.
I think image generators in general work by iteratively changing random noise and checking it with a classifier, until the resulting image has a stronger and stronger finding of “cat” or “best quality” or “realistic”.
If this classifier provides fine grained descriptive attributes, that’s a nightmare. If it just detects yes or no, that’s probably fine.
Nobody would have been looking directly at the source data. The FBI or whoever provides the dataset to approved groups, but after that you just say "use all the images in this folder" and it goes. But I don't even know if they actually provide real full-resolution images, or just perceptual hashes, or downsampled images.
And while it's possible to use the dataset to generate new images assuming the training data had full-res images, like I said, I know they investigate the people making the request before allowing access. And access is probably supervised and audited.
load more comments
(1 replies)
This is a great development, albeit with a lot of soul crushing development behind it I assume. People who have to look at CSAM or whatever the acronym is have a miserable job, so I'm very supportive of trying to automate that away from people.
Yeah, I’m happy for AI to take this particular horrifying job from us. Chances are it will be overtuned (too strict), but if there’s a reasonable appeals process I could see it saving a lot of people the trauma of having to regularly view the worst humanity has to offer without major drawbacks.
I think all CSAM should be destroyed out of respect for the victims, not proliferated. I don't care who is hanging onto this material or for what purpose.
This ain't about the victims... It never was, otherwise churches would NOT exist in current form.
This is about police and corpo state gaining power.
load more comments
(1 replies)
At this point how does it differ w/ generating AI powered CP? morons
Uh, well this one tells you if an image looks like it or not. It doesn’t generate images
If it knows if an image looks like it it can generate something like it, one step further
Correct, this kind of software is trained on CP data. So such models can be easily used to generate CP instead of recognizing it, which makes them very dangerous indeed.
Same idea as the current models that are trained to recognized cars, these models can also be used to generate a car from noise as a starting poiint.
In pretttty sure you can’t just run it in reverse like that. There’s a whole different training and operation methodology you have to use to support generating images rather than simple text classification
load more comments
(1 replies)
It differs in basically being something completely different. This is a classification model, doesn't have generative capabilities. Even if you were to get the model and it's weights, and you tried to reverse engineer an "input" that it would classify as CP, it would most likely look like pure noise to you.
Moron
load more comments
(11 replies)