Man, that sucks. It's really annoying how sycophantic these companies have made their chatbots, which makes this worse.

But, I think by the very nature of how chatbots work, you can't fix that problem. Chatbots simulate what text is likely to follow other text.

If someone online has definitely read a book that really exists, and they describe a scene they remember from that book, the clear pattern online is that someone will say "Oh yeah, that's the book X by author Y". Some people might say "I've never heard of that book", but it's likely they might just not say anything. If we're looking at a kind of forum with upvotes and downvotes, the person who found the real book will be the most upvoted.

So, that's the pattern the LLM trains on. It knows that when someone posts a question like that, the best answer is one where someone names the book and types out the passage in the book.

It would be nice if the chatbot would instead say "I don't know that book" or "That book doesn't exist".

But, it can't do that, because it doesn't "know" anything. Everything the chatbots put out is a hallucination. It's just that some hallucinations are more useful. It has been trained millions of times on patterns when the passage someone remembers is "It was the best of times, it was the blurst of times", so when it hallucinates about that sequence of words, its hallucination is bound to be more helpful. But, if what someone remembers doesn't exist, or is obscure enough that it's not in the training data, things break. Then it follows the general pattern used for every other book, and comes up with a "useful" answer that doesn't match any actual book.

If you ran it yourself you'd get a different output.

I think most of the time that's true. But, not always.

For example, I've seen people put up funny "remixes" of the "distracted boyfriend" meme. A lot of time they take historical figures, or celebrities, or characters from movies, or something that's in the news. Now, I prefer the lo-fi versions of those memes where someone just spends 2 minutes in Paint and slaps a few photos together. I've also seen GenAI versions, and occasionally they're good, even if I have an aversion to GenAI stuff.

Now, if the GenAI prompt were just something like "Distracted boyfriend meme but the boyfriend is Trump, the girlfriend is Lindsay Graham and the other girl is Putin", that might be funny, and you can just know in your head what it looks like. That's even a good lo-fi way to do it when everybody knows the meme well and can just imagine it. But, if you have a specific idea in mind that's hard to convey, it can be different.

Sometimes the prompt is hundreds of words long and has to describe the various people's clothing, their poses, their expressions, then other details like the background, foreground, lighting, etc. Sometimes in that case the result looks really good, and it's really hard to parse that all in your head.

There's an art and a skill to doing the kind of creative writing necessary to generate a good AI image. Sometimes you need to know obscure things like specific hairstyle names, car models, types of furniture, lighting techniques, facial expression details, names of articles of men's and women's clothing, how to describe poses and posture, etc. But, while that might generate an image that's exactly how you imagined, it was never designed for a human to understand, and can be really painful and/or tedious to read.

And then an elephant appears because "elephant" is a word in the prompt.

It's interesting because in some ways it's as if the AI models have a "peekaboo" level of understanding of the world.

If your prompt says something like "a man's face is hidden behind his hands", the model goes "aha, I see the word face, I better draw a face", so you have to get tricky and say "a man's hands are held up covering up the frontal area of his head".