We think in multiple passes though, we have system 1 that thinks fast and makes mistakes, and we have a system 2 that works slower and thinks critically about the things going on in our brain, that's how we correct ourselves. ChatGPT works a lot like our system 1, it goes with the most likely response without thinking, but there's no reason that it can't be one part of a multistep system that has self analysis like we do. It isn't incapable of that, it just hasn't been built yet

Can't do this YET one method to reduce this could be to: create a response to query, then before responding to the human, check if answer is insane by querying a separate instance trained slightly differently...

Give it time. We will get past this.

You're just being victim of your own biases. You only notice that was the case when you were successful in Detecting your hallucinations. You wouldn't know if you made stuff up by accident and nobody noticed, not even you.

Whereas we are checking 100% of th AI responses, do we check 100% of our responses?

Sure it's not the same thing or AI might do more,, but the problem is your example. Where people think they are infallible because of their biases. when it's not the case at all. We are imperfect, and we overlook our shortcomings possibly foregoing a better solution because of this. Because we measure the AI objectively, but we don't measure what we compare it to.