271
Nightshade - A new data poisoning tool lets artists fight back against generative AI (lemmy.world)
submitted 2 years ago* (last edited 2 years ago) by to c/technology@lemmy.ml
45 comments fedilink hide all child comments
 

The tool, called Nightshade, messes up training data in ways that could cause serious damage to image-generating AI models. Is intended as a way to fight back against AI companies that use artists’ work to train their models without the creator’s permission.

ARTICLE - Technology Review

ARTICLE - Mashable

ARTICLE - Gizmodo

The researchers tested the attack on Stable Diffusion’s latest models and on an AI model they trained themselves from scratch. When they fed Stable Diffusion just 50 poisoned images of dogs and then prompted it to create images of dogs itself, the output started looking weird—creatures with too many limbs and cartoonish faces. With 300 poisoned samples, an attacker can manipulate Stable Diffusion to generate images of dogs to look like cats.

you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 2 years ago

The idea has some merit but it's harder to implement than it looks like. Model-based image generation is heavily biased towards typical values, so you'd need a lot of poison to do it. And that poison would need to be consistent - it doesn't work if you tell the model now that cats are dogs and then that ferrets are dogs, you need to pick one.

I'm rather entertained by the amount of fallacies and assumptions ITT though. I get that you guys are excited with model-based image gen; frankly, I'm the same when it comes to text gen. But those two things won't help, learn the difference between "X is true" and "I want X to be true".

  • permalink
  • fedilink
  • source
    •