Tech experts are starting to doubt that ChatGPT and A.I. ‘hallucinations’ will ever go away: ‘This isn’t fixable’::Experts are starting to doubt it, and even OpenAI CEO Sam Altman is a bit stumped.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 3 years ago (3 children)

I did manage to write a back-propogation algorithm, at this point I don't fully understand the math behind back-propogation. Generally back-propogation algorithms take the activation, calculate the delta(?) with the activation and the target output (only on last layer). I don't know where tokens come in. From your comment it sounds like it has to do something in a unsupervised learning network. I am also not a professional. Sorry if I didn't really understand your comment.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 3 years ago (2 children)

    Mathematically, I have no idea where the tokens come in exactly. My studies have been more conceptual than actually getting down to the knitty-gritty, for the most part.

    But conceptually, from my understanding, tokens are just a variable that is assigned a speculated fitness, then used as the new "base" data set.

    I think chicken would go good in this, but beef wouldn't be as good. My token is the next ingredient i am deciding to put in.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 3 years ago (1 child)

    You guys should all check out Andrej Karpathy's neural networks zero to hero videos. He has one on LLMs that explains all this.

  • source
  • parent
  • hideshow 1 child comment