679
AI in reality (slrpnk.net)
submitted 2 years ago* by to c/programmer_humor@programming.dev
113 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 11 points 2 years ago (5 children)

While I haven't read the paper, the comment's explanation seems to make sense. It supposedly contains a mathematical proof that making AGI from a finite dataset is a NP-hard problem. I have to read it and parse out the reasoning, if true, it would make for a great argument in cases like these.

https://lemmy.world/comment/14174326

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 2 points 2 years ago (4 children)

    If that is true, how does the brain work?

    Call everything you have ever experienced the finite dataset.
    Constructing your brain from dna works in a timely manner.
    Then training it does too, you get visibly smarter with time, so on a linear scale.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 9 points 2 years ago* (3 children)

    I think part of the problem is that LLMs stop learning at the end of the training phase, while a human never stops taking in new information.

    Part of why I think AGI is so far away is because to run the training in real-time like a human, it would take more compute than currently exists. They should be focusing on doing more with less compute to find new more efficient algorithms and architectures, not throwing more and more GPUs at the problem. Right now 10x the GPUs gets you like 5-10% better accuracy on whatever benchmarks, which is not a sustainable direction to go.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 2 years ago (2 children)

    How does conversation context work though? Is that memory not a form of learning?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 2 years ago* (1 child)

    The context window is a fixed size. If the conversation gets too long, the start will get pushed out and the AI will not remember anything from the start of the conversation. It's more like having a notepad in front of a human, the AI can reference it, but not learn from it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 2 years ago

    Also, a key part of how GPT-based LLMs work today is they get the entire context window as their input all at once. Where as a human has to listen/read a word at a time and remember the start of the conversation on their own.

    I have a theory that this is one of the reasons LLMs don't understand the progression of time.

  • permalink
  • fedilink
  • source
  • parent