938
Surely they yearn for the mines (thelemmy.club)
submitted 1 day ago by to c/lemmyshitpost@lemmy.world
91 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 15 hours ago (1 child)

Current LLMs are limited by the fact that they can't train during inference. i.e. They can't learn. If we find something better than backpropagation or diffusion, then things are going to actually be different. At that point, the models would be able to compose ideas together instead of just predict the next token in sequence based on previous training.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 15 hours ago

    Current LLMs are limited by the fact that they can’t train during inference. i.e. They can’t learn

    Learning is much more than just training during inference.

    True unsupervised learning is still in its infancy, and requires a completely different architecture than the current fancy auto-suggest boom. Unsupervised learning has been the dream since at least the 1950s when AI was an aspiration, even before transistors existed. Nobody has figured out how to do it yet.

    If pigs learn to fly, then things are going to actually be different.

  • permalink
  • fedilink
  • source
  • parent