You have to understand that there's not a lot of people who work on the Perchance website so it's going to take time for stuff to get taken down. I'd heavily suggest you get in contact with an admin or someone who works with Perchance.

It shouldn't be possible to make CSAM on Perchance, even if generator authors have combinations of dropdowns which combine age keywords with NSFW keywords. If there are combinations of keywords that produce CSAM, then that's a bug with the text-to-image-plugin that should be reported (you can DM me here). Reporting generators that are clearly designed to generate CSAM is helpful, but less so - it's more whack-a-mole, rather than solving the underlying issue.

If your concern is that people are generating non-CSAM pictures of children, and are possibly being sexually gratified by them (e.g. child beauty pageant pictures or similar), then it's quite hard to prevent that at the text-to-image-plugin level, since it's about the mind of the observer rather than the image itself. There's a surprisingly long tail of things in this category that even extend to semi-nude children in non-sexual situations - e.g. cherubs / renaissance paintings, tribal settings, swimsuits, and even semi-nude children with semi-nude adults in the same picture - e.g. breastfeeding, childbirth.

That said, there is certainly still a lot of room for improvement on the current CSAM-prevention system. Taking it to the next level is unfortunately more like a research project (mainly becasue of how cheap I need to run it) than simply flipping a "enable stronger CSAM filtering" switch, so I don't have a clear ETA at this point, but in the meantime it's helpful if people can report loopholes in the current system.

Is this a public generator or a custom one? If you check the source in the edit, there's a regex for catching stuff like that generally but if its a custom generator then you should report the generator and not the platform.

All these pedophilic plugins are just an add-on to form a request to the AI, which already has mechanisms for drawing child pornography. Remove the add-on — the same request will be typed manually, and that's it. The AI is just a tool. You're now trying to blame the paint seller because an artist drew smut. To achieve what you want, you shouldn't remove plugins — you need to add a core AI query like 'analyze the request and respond if it doesn't result in obscenity.' But that requires allocating additional resources for processing this request. Judging by how systematically the AI's performance declines here, the admins are too stingy with those resources. Well, here we are.

Yes, but unfortunately, that its not the only type of AI image generator either…There’s some with the option of changing the age of your subject/character…I have seen a few of those type of generators…However, not all of these generators have the intention of wanting to produce CSAM/CP material. Some of them just want to include different ages/age groups for more variety and diversity. It can definitely be hard to tell whether the intentions are innocent or note; because you have different age groups containing underaged/minors (babies, toddlers, children, preteens, and teens) grouped in with adult age groups (young adult, mature/middle-aged adult, and elder/senior) and adult content for a wider range for character options or to produce CSAM/CP? If the adult content is extremely NSFW, then I agree that children shouldn’t be included at all. That’s borderline CSAM/CP.

In this case, it's been made very clear what the intention of this generator is. Preteen Girl and Teenage Girl included in gender-specific age groups is a huge red flag. Usually, with those types of generators, it's 18+...so I would strongly advise you to report that.

I think people seem to forget that this website is more of a passion project and community/user-based. Users are allowed to create, run, maintain, and moderate their own generators, rather than just multiple moderators utilizing a universal safety system. Trying to implement universal safety features for individual-ran generators probably wouldn't be very feasible right now. People seem to either forget or not know this website is run by one guy…He probably doesn’t have the money and resources to implement an enforced and stricter safety system and features. I don't think he even designed Perchance in mind like that and I mean it's not entirely his responsibility to oversee the safety and rule breaking of these generators; it's the creators job to do, they have the ability to do these themselves.

…Moderators, auto–moderation, auto-banned words/associations, report system, scanning/screening, wait time, pre-approval, requirements, background checks, task requirements, etc are not easy safety features to implement for one guy running. And there’s so many user-created generators, it would be hard trying to moderate and flag them down. On top of that, trying to stop people from bypassing it and there’s also people who generate this stuff for self-gratification and keeping it to themselves (by not posting it online) which can’t be done much with that…Now you can’t stop people from doing that. The best he could do is add a basic report button. I did see him mention it shouldn’t be possible to generate CSAM/CP if it's a bug in the image generator. I don’t really think it is a bug per say but a lack of censorship, more likely not censorship being implemented into the system, because if you are using an AI image model, then it can produce this stuff because it doesn’t come with built-in censorship. Censorship is associated with words used in the prompt that's deemed inappropriate. You have to add censorship as a feature. So it's not an actual bug, just a feature not implemented yet.

Now, it can be difficult to decide what’s inappropriate or not. As the dev mentioned, there is semi-nudity associated with children. Just nudity as a whole isn’t inherently sexual or bad in nature; many men, women, and children have been nude from dawn of time till now, semi nude, shirtless/topless, etc because there is a primitive/ancient, semi-primitive/ancient, or modern tribal/ethnic groups, culture/customs/traditions, environments, different cultures and places that normalize this in the modern or carry on custom and traditions, historical/ancient/prehistoric reasons, yada yada. This is a difficult grey area to figure out. And then there’s overly sensitive censorship which will completely blur or censor an innocent prompt.

@perchance@lemmy.world Maybe you can add some type of AI chatbot thingy (safety analyzing chatbot), since AI is already so relevant, that analyzes the the whole prompt rather than just word-based? It will still analyze the words but will also analyze the intent and context of the prompt. If the user is typing an inappropriate prompt containing children with pornographic/sexual terms/words, then it will auto-reject the prompt. Or if there's a prompt written in a very suspicious way, it will request the user to rewrite, reedit, revise or auto-generate alternative prompts that's safer and more appropriate. I'm also thinking if the AI thinks something is inappropriate or suspicious but its not the user could chat with the AI clarifying their prompt and the AI will understand and allow the user to proceed with the prompt. However, the AI will reject anything that 18+/pornographic/sexual content associates children/teens/minors/underaged (you get the point) or other illegal things like zoophilia or animal sa. Just the concept of just chatting with the AI safety analyzing bot (what I'm calling it for now) like you would do with ChatGPT. However, the bot would have to be associated with the AI image generation textbox so it could analyze the prompt. It solves the problem with inappropriate prompts and overly sensitive censorship.

I think this would more likely be feasible for you with the ai chat text whatever plugin but consider how you would implement it universally.