▲ 190 ▼ Open-source Deepseek R1 dethrones commercial AI, now allegedly being hit by cyberattack (www.cnbc.com) submitted 2 years ago by JOMusic@lemmy.ml to c/world@lemmy.world 50 comments fedilink hide all child comments
[–] Pieisawesome@lemmy.world 4 points 2 years ago It’s because LLMs don’t work with letters. They work with tokens that are converted to vectors. They literally don’t see the word “strawberry” in order to count the letters. Splitting the letter probably separates them into individual tokens permalink fedilink source parent