502
We have to stop ignoring AI’s hallucination problem (www.theverge.com)
submitted 2 years ago by to c/technology@lemmy.world
204 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 2 years ago* (last edited 2 years ago) (4 children)

I'm not sure where you think I'm giving it too much credit, because as far as I read it we already totally agree lol. You're right, methods exist to diminish the effect of hallucinations. That's what the scientific method is. Current AI has no physical body and can't run experiments to verify objective reality. It can't fact check itself other than be told by the humans training it what is correct (and humans are fallible), and even then if it has gaps in what it knows it will fill it up with something probable - but which is likely going to be bullshit.

All my point was, is that to truly fix it would be to basically create an omniscient being, which cannot exist in our physical world. It will always have to make some assumptions - just like we do.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 2 years ago* (1 child)

    The fundamental difference is that the AI doesn't know anything. It isn't capable of understanding, it doesn't learn in the same sense that humans learn. A LLM is a (complex!) digital machine that guesses the next most likely word based on essentially statistics, nothing more, nothing less.

    It doesn't know what it's saying, nor does it understand the subject matter, or what a human is, or what a hallucination is or why it has them. They are fundamentally incapable of even perceiving the problem, because they do not perceive anything aside from text in and text out.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 2 years ago*

    Many people's entire thought process is an internal monologue. You think that voice is magic? It takes input and generates a conceptual internal dialogue based on what it's previously experienced (training data for long term, context for short term). What do you mean when you say you understand something? What is the mechanism that your brain undergoes that's defined as understanding?

    Because for me it's an internal conversation that asserts an assumption based on previous data and then attacks it with the next most probable counter argument systematically until what I consider a "good idea" emerges that is reasonably vetted. Then I test it in the real world by enacting the scientific process. The results are added to my long term memory (training data).

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 years ago (1 child)

    It doesn't need to verify reality, it needs to be internally consistent and it's not.

    For example I was setting up logging pipeline and one of the filters didn't work. There was seemingly nothing wrong with configuration itself and after some more tests with dummy data I was able to get it working, but it still didn't work with the actual input data. So I have the working dummy example and the actual configuration to chatGPT and asked why the actual configuration doesn't work. After some prompts going over what I had already tried it ended up giving me the exact same configuration I had presented as the problem. Humans wouldn't (or at least shouldn't) make that error because it would be internally inconsistent, the problem statement can't be the solution.

    But the AI doesn't have internal consistency because it doesn't really think. It's not making sure what it's saying is logical based on the information it knows, it's not trying to make assumptions to solve a problem, it can't even deduce that something true is actuality true. All it can do is predict what we would perceive as the answer.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 2 years ago*

    Indeed. It doesn't even trend towards consistency.

    It's much like the pattern-matching layer of human consciousness. Its function isn't to filter for truth, its function is to match knowns and potentials to patterns in its environment.

    AI has no notion of critical thinking. It is purely positive "thinking", in a technical sense - it is positing based on what it "knows", but there is no genuine concept of self, nor even of critical thinking, nor even a non-conceptual logic or consistency filter.

  • permalink
  • fedilink
  • source
  • parent