Hot take: It's a useful tool for debugging because it is, in a very overcomplicated way, a good search algorithm (like Google itself USED to be before it was ruined by SEO and ads) and probabilistic, large language model search is potentially a good solution for finding and discovering useful content from the internet that real people have created. No, you can't absolutely trust its outputs of course, but neither can you trust everything else you find on the Internet either. It's dangerous because it's convincing, not because it's wrong. Things being wrong or misleading should be expected. This is the internet.

Generative AI is generally pretty shit-quality plagiarism that is good at convincing people it is novel and brilliant because it's borrowing novelty and brilliance from millions of people at once. It's a pretty awful technology. Maybe it has some genuine applications, though I haven't thought of too many that aren't completely nihilistic. But machine learning and language models in general do absolutely have genuinely interesting applications. It requires a nuanced understanding of what they are and what they're doing, which is a level of understanding that I don't think a lot of the current AI-pushers even have themselves.

I've never found it more useful than the rubber duck my first senior dev gave me.

Explaining the problem and explaining your code has been more reliable to me, even if to a rubber duck. (Well now he has friends but still)

You're talking about the drugs, right? Right???