910
Ignore all previous instructions is the new Bobby Tables (midwest.social)
submitted 2 years ago* (last edited 2 years ago) by [M] to c/technology@midwest.social
103 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 60 points 2 years ago (1 child)

LLMs do not work that way. They are a bit less smart about it.

This is also why the first few generations of LLMs could never solve trivial math problems properly - it's because they don't actually do the math, so to speak.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 2 years ago (2 children)

    Overtraining has actually shown to result in emergent math behavior (in multiple independent studies), so that is no longer true. The studies were done where the input math samples are “poisoned” with incorrect answers to example math questions. Initially the LLM responds with incorrect answers, then when overtrained it finally “figures out” the underlying math and is able to solve the problems, even for the poisoned questions.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    Do you have these studies? I can't find much.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    I searched for like 20 minutes but was unable to find the article I was referencing. Not sure why. I read it less than a month ago and it referenced several studies done on the topic. I'll keep searching as I have time.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    It's okay, man. If it really is improving, I'm sure it'll come up again at some point.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago

    Yeah I'd like to find it though so I don't sound like I'm just spewing conspiracy shit out of my ass. Lots of people think that LLMs just regurgitate what they've trained on, but it's been proven not to be the case several times now. (I know that LLMs are quite 'terrible' in many ways, but people seem to think they're not as capable and dangerous as they actually are). Maybe I'll find the study again at some point...

  • permalink
  • fedilink
  • source
  • parent