you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 1 year ago (2 children)

I'm surprised it managed to do addition correctly.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 year ago (1 child)

    Additional is fairly trivial for a neural network to learn.

    Weight 1 plus weight 2 equals output is literally the baseline model structure.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 5 points 1 year ago

    It's actually a fairly involved process because the tokens representing 1 and 4 don't have any mathematical correlation with the numbers 1 and 4 so you can't math them directly to get to 5.

    Apparently how they do it is by a series of approximations from big numbers to small numbers, not too dissimilar from the way a human would do it. The anthropic team published a paper about it recently, I can dig it up if you're interested.

  • source
  • parent