There seems to be legal discussions about that. It’s not quite as simple as you say:

However, there may be cases in which a different assessment is justified, namely when users use and operate the LLM as a tool that merely implements their personal creative intent. This could be compared somewhat more vividly to using a paintbrush. If the brush merely rolls over the paper, for example because it is dropped, no copyright-protected work is created, even if paint remains on the paper. However, if a painter deliberately swings the brush in a certain way, a protected painting can be created. If AI is used in a comparable way a copyright-protected work can indeed be created.

https://kpmg-law.de/en/ai-and-copyright-what-is-permitted-when-using-llms/