Thought today's smbc comic was too funny not to share. Source

you are viewing a single comment's thread
view the rest of the comments
[–] -4 points 3 days ago (4 children)

? A neural network is not a lookup table.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 11 points 3 days ago* (2 children)

    it's a compressed lookup table. rather than there being one response for every input, the input is used as a seed to decompress relevant parts of the dataset, with some added randomness. you can even do it with gzip itself: https://nathan.rs/posts/gzip-lm

    the linked paper is very good: https://arxiv.org/pdf/2309.10668

  • source
  • parent
  • hideshow 2 child comments
  • [–] 6 points 3 days ago

    It could be represented with a series of lookup tables, especially quantized LLMs. A series of inputs results in a specific output that gets passed to the next set of nodes. Repeat 7 billion times, and the final output from the last set of nodes is a set of token probabilities.

  • source
  • parent