I think that it's more simple to just say that LLMs are highy advanced electronic parrots - they put sentences together out of sentences that they've been exposed to before, the more often they've been exposed to them the more likely they'll produced it.

LLMs and Parrots are both just copying stuff without understanding it, it's just that the LLM has a very deep multi-level mathematical engine to find correlations between words and even between word assemblies of various sizes, all in much larger windows of text (literally within tens of tousands and even hundreds of thousands of words) whilst the Parrot's "correlation detection" isn't mathematically strict or all that deep or wide (about enough to output "Polly wants a cracker" if they hear the word "cracker").

Curiously, of both the Parrot is the one which actually has some cognition (i.e. they think), whilst the LLM is a purelly statistical engine.

IMHO the whole use of "hallucination" just muddles the waters because when we hear the word "hallucination" we think the human version of it (seeing that which isn't there) which makes us think of human cognition because it's generally a cognitive problem, whilst the LLM hallucination is something very different - producing a seemingly truthful piece of text which is in fact not truthful.

LLMs don't hallucinate in the traditional sense of the word because as unthinking statistical text analysis and production engines they have no cognition to malfunction in that way.

So in a way by using the word "hallucination" for LLMs we are indirectly supporting in people's minds the false idea that LLMs have cognition.

Highly fluent crap is what I get a lot from chatgpt.