The tool, which is called Kromix, bills itself as an “AI image styler.” A voice-over for one of the video advertisements, viewed by WIRED, promotes the tool as having “no restrictions,” saying “all the characters are real people” and describing it as “the AI that men actually use.” In one ad, a woman closely resembling a prominent female US politician is seen in front of a US flag and the flag of the president of the US above the caption “What if she moved?” She then winks, before the ad cuts to a scene of a woman with the same face performing in a pornographic video.

The Kromix app, which was hosted by Apple’s App Store prior to a request for comment from WIRED, invites paid users to upload photos so that it can use them in scenarios including “bedroom rape” and “Disney love.” Videos on the app also feature AI-generated pornographic videos of women wearing Spider-Man costumes. Requests for comment sent to a contact listed in the app went unanswered.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 4 days ago* (1 child)

I can't help but think the near complete lack of moderation at meta is intentional considering they must be making ad revenue of this and other controversial aps.

Still, the bigger story is that this is happening at all, that it's all women, and it's all geared toward men. We can assume it's selling if there's this much of it, that it's this hard to keep off platforms, and that's fucking horrifying.

  • source
  • hideshow 2 child comments
  • [–] 1 point 4 days ago (1 child)

    There's plenty of moderation. It's just not doing what you'd like it to.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 4 days ago (1 child)

    Paul says that Meta has increasingly relied on AI to flag and vet advertisements, while laying off human moderators. “Ads are Meta’s primary product, and there appears to be zero quality control,” she says. “Meta is also making a lot of statements publicly, banking on their AI and the sophistication of it, while simultaneously saying they use AI to review their ad library and nothing's perfect.”

  • source
  • parent
  • hideshow 2 child comments