you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 years ago (7 children)

Not really a concern. It's basically translation, which language models excel at. It just needs a mapping of the hex to byte

  • source
  • parent
  • hideshow 7 child comments
  • [–] 2 points 2 years ago (6 children)

    It is a concern.

    Check out https://tiktokenizer.vercel.app/?model=deepseek-ai%2FDeepSeek-R1 and try entering some freeform hexadecimal data - you'll notice that it does not cleanly segment the hexadecimal numbers into individual tokens.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 1 point 2 years ago (5 children)

    I'm well aware, but you don't need to necessarily see each character to translate to bytes

  • source
  • parent
  • hideshow 5 child comments
  • [–] 1 point 2 years ago (4 children)

    It's not out of the question that we get emergent behaviour where the model can connect non-optimally mapped tokens and still translate them correctly, yeah.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 2 years ago (3 children)

    I’m confused, is the concern when the model doesn’t properly identify when it is using software to identify something like a hex pattern?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 2 years ago (2 children)

    The concern is that the model doesn't actually see the world in terms of distinct hexadecimals, but instead as tokens of variable size - you can see this using the tiktokenizer-webapp: enter some text and it will split it into the series of tokens the model actually will process.

    It's not impossible for the model to work it out anyway, but it is a reason for this type of task to be a bit harder on LLMs.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 years ago (1 child)

    I understand how base models tokenize language. What I’m curious about you’re basing your response off a horrendously screenshotted meme image of someone interacting with deepseek. Is your concern that deepseek isn’t showing the code used to approach a hex string? Because that’s certainly a valid concern, though you can ask the model to output the code it is running. That’s definitely an ethics improvement that should be made in the UI, but it’s very clear what the model is doing under the hood

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 2 years ago

    I don't think DeepSeek has the capability of generating code and executing it inline in the context window to support its answers, in the way that ChatGPT does - the "used"-part of that answer is likely a hallucination, while "or would use" more accurately represents reality.

  • source
  • parent