363
submitted 16 hours ago* (last edited 16 hours ago) by technocrit@lemmy.dbzer0.com to c/fuck_ai@lemmy.world

In March, a girl’s stepfather took his own life after cops discovered that he had used Grok to create 7,000 sexually explicit images using one photo taken when his stepdaughter was 11 years old, the amended complaint alleged.

Grok allowed the man to generate extreme images depicting incest and rape without flagging any harmful behavior, the complaint said. Seemingly, xAI’s child safety system only intervened after the man input a prompt for “gang rape.” That request sent a CyberTip to the National Center for Missing and Exploited Children (NCMEC), which alerted law enforcement to the AI CSAM.

Yet the harm was not stopped then, either. Despite mandatory reporting requirements to share information like a user’s IP address when CSAM is flagged, xAI repeatedly refused to help cops or NCMEC identify the user, the complaint alleged. For weeks, xAI allegedly “obstructed this investigation at every turn” and made it harder for “law enforcement efforts to locate, identify, and apprehend the perpetrator.”

Eventually, the stepfather was arrested after cops obtained a warrant to seize his devices. That’s when “a forensic review revealed approximately 7,000 AI-generated images and videos” depicting his stepdaughter, which were allegedly produced using Grok. Without Grok providing users with easy access to “undressing” capabilities, his family doubts he ever would have generated the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.”

top 50 comments
sorted by: hot top new old
[-] Yerbouti@sh.itjust.works 12 points 5 hours ago

Nothing dystopic about this headline.

[-] Mulligrubs@lemmy.world 11 points 6 hours ago

On one hand, this is great news, he's no longer here to be gross.

On the other hand, could these images have served as an outlet for his perversion, and kept his step-daughter safe for years?

On the other other hand, was access to this what twisted him? Was he reasonable at one time?

He did the right thing in the end, this was the best possible outcome for everyone.

[-] Alvaro@lemmy.blahaj.zone 6 points 1 hour ago

Geez, if only there were alternatives to ai generated csam, like therapy and medication??

Yes, it's not a perfect fix, and it is hard, but people overcome drug addictions with therapy and meds, and I am pretty sure that nothing beats heroin...

[-] starelfsc2@sh.itjust.works 2 points 1 hour ago

Is there evidence therapies and medication actually help their desires for CSAM? I don't know if there's great research in either direction

[-] Mycatiskai@lemmy.ca 8 points 2 hours ago

He was trading these pictures for material from others so other children were hurt in trade of his step child being victimized. She may not have been physically victimized but her likeness was portrayed in CSAM which now she knows about and is going to damage her mentally.

[-] FE80@lemmy.world 6 points 5 hours ago

The future is fuckin weird.

[-] banazir@lemmy.ml 1 points 1 hour ago

The future is what we make of it. Unfortunately, we're kind of shit.

[-] blazeknave@lemmy.world 7 points 7 hours ago

I used to sell trust and safety and it was scary then. Now they've fired those teams and I'm pretty sure there's zero moderation on the Internet besides Reddit

[-] AmbitiousProcess@piefed.social 62 points 11 hours ago

Every image Grok generates of CSAM is another reason to rightfully call Elon a pedophile for knowingly allowing it to happen.

[-] Tartas1995@discuss.tchncs.de 2 points 52 minutes ago

I really hope France fucks them up. In France, there is no question about the illegal status of fictional csam, it is illegal.

[-] Duamerthrax@lemmy.world 24 points 10 hours ago
[-] FoxAlive@lemmy.zip 6 points 4 hours ago* (last edited 4 hours ago)

Those idiots think this should be legal, but not when the civilians do it. Tim Sweeny probably is only piping up because he insist on posting the stupidest thing his brain can come up with every day.

In no way shape or form has elon tried to dampen or stop this from happening to this day. He just made it so only paid users not in the united States can keep producing csam.

And trump wants this. This is the mysterious Ai progress we keep hearing about, next to price fixing, and survalience. States are not allowed to regulate Ai, and people will tell you that you are a dangerous person for trying to suggest that elon musk should be held accountable.

We are not all here looking for the guy who did it or confused on why this is happening. Elon musk has created a csam generator and doesnt have to stop it. Instead he just occasionally has to send one of his super fans to prison for using it the way he intended to. All to say hes doing something to stop this problem.

7k images and videos is a lot to go unnoticed. Its almost like this is allowed on their platform and encouraged. This isn't a "we can't be held accountable for what our users post" problem. This is a elon musk left a loaded gun out on a table, and he should be held liable for not putting it in a gun safe. Especially when every single other mainstream llm doesnt have this problem. Its negligence to the point of being willfully ignorant. You can't tell me they can ban you for a single comment about Israel but all the sudden their moderation isn't the greatest when 7k images and videos where being produced?

[-] SatansMaggotyCumFart@piefed.world 37 points 12 hours ago

Seven thousand sex images of an eleven year old is a lot.

[-] defaultusername@lemmy.dbzer0.com 46 points 11 hours ago

One sex image of an eleven year old is a lot.

[-] Damage@feddit.it 13 points 11 hours ago

I mean, I guess he was trying to get one with the right number of fingers and human eyes

load more comments (4 replies)
[-] lemmyng@lemmy.world 4 points 8 hours ago

One day Elon will go out the same way this pedophile did.

[-] maniajack@lemmy.world 123 points 15 hours ago

All the people who bought SpaceX stock invested in this. Grok is owned by SpaceX. It would be nice if journalists would put that in their articles.

[-] wonderingwanderer@sopuli.xyz 7 points 8 hours ago

The CSAM generator is in the index funds 🤢

[-] BarneyPiccolo@lemmy.today 16 points 11 hours ago

If you're driving a Tesla, you are funding CSAM.

[-] chiliedogg@lemmy.world 51 points 15 hours ago

Lawsuits for this need to start including shareholders as defendants. Make investing in it a civil and reputational liability.

load more comments (3 replies)
load more comments (2 replies)
[-] honesthenery 5 points 8 hours ago
[-] NikkiDimes@lemmy.world -1 points 1 hour ago

Honestly? I hear you, but I'm going to have to push back on that

[-] victorz@lemmy.world 18 points 11 hours ago

In one way, I'm glad he produced fake images of his stepdaughter, rather than force her to produce real photographs.

But this part is still very upsetting:

the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.”

[-] Duamerthrax@lemmy.world 15 points 10 hours ago

If those images could be made without a sample set of real CSAM, I probably wouldn't have as much of an issue with it, but some child suffered for that. In addition, they were intentionally based on a real child's face and that could come back to the child, makes it even worse.

load more comments (4 replies)
[-] TachyonTele_Esq@piefed.social 148 points 16 hours ago

This entire thing from start to finish is horrifying.

[-] Marthirial@lemmy.world 75 points 15 hours ago

Yes, from Elom being born all the way to buying Twitter.

[-] cecilkorik@lemmy.ca 39 points 15 hours ago

The story was probably pretty horrifying before Elon was born too, from what we already know of his father.

load more comments (1 replies)
[-] Wispy2891@lemmy.world 12 points 11 hours ago

More than half of all traffic flowing through Grok, Elon Musk's flagship AI product, now comes from users requesting pornographic images, explicit videos, and erotic roleplay — and a new investigation reveals that xAI made this outcome a deliberate strategic choice

https://www.techtimes.com/articles/319142/20260626/grok-adult-content-tops-10-billion-images-monthly-xai-engineers-admit-csam-has-no-fix.htm

[-] Cytobit@piefed.social 12 points 11 hours ago

xAI repeatedly refused to help cops or NCMEC identify the user

Well, of course xAI refused to help. It's not like they are one of the victims here, they are accomplices to the crime.

[-] 0li0li@lemmy.world 44 points 14 hours ago

A major company makes CSAM generation platfor and sharing network, yet it does not get seized or closed by any government entity. Makes sense.

load more comments (1 replies)
[-] limonfiesta@lemmy.world 31 points 13 hours ago

There's a reason why Ed Zitron refers to Grok as the CSAM generator.

[-] Mediocre_Bard_Redeux@lemmy.world 72 points 15 hours ago

Jesus Fucking Christ.

Just turn it off at this point.

[-] gedfromgont@piefed.ca 3 points 2 hours ago

You expect this dude to switch it off?

[-] ImgurRefugee114@reddthat.com 16 points 12 hours ago

I don't think Elon want to turn off the CSAM creation machine, otherwise he probably would have some time between when he instructed the construction of the CSAM creation machine and now.

[-] _fryerDan@sh.itjust.works 19 points 12 hours ago

why are pedophiles so prevalent in todays society? jfc

[-] Imgonnatrythis@sh.itjust.works 8 points 10 hours ago

It helps people sleep better at night to think all these people are satanic monsters, but it's a mental illness that no one wants to admit to or talk about. Thankfully most of the afflicted control their thoughts and never take any horrific action. With gateways like Grok lowering the bar for experimentation with these types of ill thoughts, more people are starting to act in ways that bring this to attention. All technology can be used for bad things, but the good things these generative AIs can do are really so miniscule that there's no reason not to place strong legislative restrictions on them right now to limit abuse. That and maybe try to do something about the mental health crisis ravaging the US (and frankly much of the world right now).

load more comments (4 replies)
[-] Mouselemming@sh.itjust.works 8 points 10 hours ago* (last edited 10 hours ago)

He killed himself because he got caught, not from any kind of remorse. If only Grok would have done the same.

(Obviously no CSAM would have been better but Grok's self-destruction would at least prevent more of the same, and take $ out of Elon's pocket)

[-] Rivermoonwolf@lemmy.world 23 points 13 hours ago

Every time a Republican says "think of the children!" I usually respond with "you are. With your hand in your pants".

Burn it to mother fucking ground. Then salt the earth.

[-] trslim@pawb.social 3 points 8 hours ago

that was quite a headline to be scrolling thru

[-] defaultusername@lemmy.dbzer0.com 8 points 11 hours ago* (last edited 11 hours ago)

I don't know which one is more disturbing, the pedophile stepfather or the company that allows these kinds of pictures to be generated in the first place. Though Elon is also a pedo who begged Epstein to invite him to his pedo island, so...

load more comments (2 replies)
load more comments
view more: next ›
this post was submitted on 09 Jul 2026
363 points (98.9% liked)

Fuck AI

7589 readers
1831 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

founded 2 years ago
MODERATORS