you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 2 years ago

It’s because LLMs don’t work with letters. They work with tokens that are converted to vectors.

They literally don’t see the word “strawberry” in order to count the letters.

Splitting the letter probably separates them into individual tokens

  • source
  • parent