LLMs are sort of like a bastard child outgrowth of machine learning systems. They aren't necessarily hot garbage -- e.g., DeepSeek is surprisingly good at scraping and summarizing research papers that have been fed into it -- but the general-purpose commercial shit like ChatGPT, Bard, Copilot, etc. are all more or less snake oil. Doing this stuff with manuals, technical whitepapers, and so forth is probably viable, but once they started feeding in social media posts -- even StackOverflow -- they lost the plot entirely. I say this as a senior-level software developer who constantly has to look shit up because I bounce between too many languages/platforms and nearly always need a refresher for whichever one I am currently trying to beat into submission, and I have been skimming the Google AI overview shit and checking its work more often than not lately. It doesn't always understand what I am looking for and tries to shit something out of left field anyway, so I just go straight to whatever Reddit/Stack/Baeldung/NerdRanch links pop up in the first few pages of results, but when it does "get" it, it's pretty close. If it's doing anything more than summarizing a Linux man page, I still click through to its source links because the highest-updooted answer (which is what the overview uses) isn't always the most correct for what I'm doing.

Regarding other actual uses, I think it was @microfiche@hexbear.net a few weeks back that had an anecdote about using one of the public LLMs to lay out some plumbing plans for a residential space given a set of codified rules, and the slop machine came out surprisingly close to the mark because it had so many guard rails around it (due to the building/plumbing code). I could see it working as a sanity check for tradespeople, civil engineers, and architects if they're running specialized models and the LLM is ultimately just a user interface. We're not really there in the US though. Shit's too unregulated and consumes entirely too much energy for what is still ultimately a novelty.