The TL;DR is that there is a voluntary system that tracks court cases and will alert CSAM victims when it suspects images of them (either known images, or new ones) are the topic of criminal investigations. A Jane Doe was repeatedly raped in the early 2000’s to produce copious amounts of CSAM, which was broadly shared among pedophile groups. Her images were apparently very popular and prolific. Doe was alerted that Grok was producing new explicit images of her as a child. The images in question are novel (generated by AI, and not matching any known existing images) but are undoubtedly of Doe as a child. Meaning it was using Doe’s images as part of the training data for the images it generates.

Essentially, if Grok was trained on CSAM, Doe’s popular images were almost certainly included. And now Grok is producing new images of her. It would be like asking Grok to make an image of a popular egirl, and getting an image of what is undoubtedly Belle Delphine. And then X goes “no no no, we didn’t use any images of Belle Delphine in our training data. We promise!”