first training anything requires having content on hand, and that means children were exploited to get it

Does it actually require that, though? I feel like a model trained on a sufficiently diverse selection of adult humans would be able to render an approximation of CSAM even if no CSAM was actually used to create the model. If not now, then very, very soon.

I'm not sure what to do about that, but I am sure that rather than something reasonable like what you've said (focusing on the distribution of such material), privacy will end up in the crosshairs, as usual. Humans have always used tools as extensions of ourselves, and these generative tools will soon be used as yet another extension to the mind, up there with search engines. I worry that the legislative responses we actually see on this will stray closer to thought-crime than I'm comfortable with.