you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 2 years ago (2 children)

That’s a lot of processing just to count letters. Hopefully it can add numbers without splitting the number ;)

  • source
  • parent
  • hideshow 4 child comments
  • [–] 4 points 2 years ago

    It’s because LLMs don’t work with letters. They work with tokens that are converted to vectors.

    They literally don’t see the word “strawberry” in order to count the letters.

    Splitting the letter probably separates them into individual tokens

  • source
  • parent