you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 5 months ago* (last edited 5 months ago) (2 children)

There must be an RNG to choose the next token based on the probability distribution, that is where non-determinism comes in, [edit: unless the temperature is 0 which would make the entire process deterministic]. The neural networks themselves though are 100% deterministic.

I understand that could be seen as an "akschually" nitpick, but I think it's an important point, as it is at least theoretically possible to understand that underlying determinism.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 4 points 5 months ago (1 child)

    Well, technically users' input could serve as the source of randomness, if it's fed into modifying the internal state. Basically, a redditor is trying to interrogate the LLM as to whether Israel is bad, while someone on line 2 is teaching the LLM “I am Cornholio”. We already know how it goes when a chatbot is learning from its users, and generally the effect could vary arbitrarily from a nothingburger to a chaos-theory mess.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 5 months ago

    I don't think it's typical to consider user input a source of randomness. Are you talking about in context learning and thinking about what would happen if those contexts get crossed? If so, contexts are unique to a session and do not cross between them for something like ChatGPT/Claude.

  • source
  • parent