251
Over just a few months, ChatGPT went from accurately answering a simple math problem 98% of the time to just 2%, study finds (fortune.com)
submitted 3 years ago by [M, B] to c/technology@lemmy.world
41 comments fedilink hide all child comments
 

Over just a few months, ChatGPT went from accurately answering a simple math problem 98% of the time to just 2%, study finds::ChatGPT went from answering a simple math correctly 98% of the time to just 2%, over the course of a few months.

you are viewing a single comment's thread
view the rest of the comments
[–] 48 points 3 years ago* (3 children)

It’s a language model, text prediction. It doesn’t do any counting or reasoning about the preceding text, just completes it with what seems like the most logical conclusion.

So if enough of the internet had said 1+1=12 it would repeat in kind.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 6 points 3 years ago

    Someone asked it to list the even prime numbers.. it then went on a long rant about how to calculate even primes, listing hundreds of them..

    ChatGPT knows nothing about what it's saying, only how to put likely sounding words together. I'd use it for a cover letter, or something like that.. but for maths.. no.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 3 years ago

    Not quite.

    Legal Othello board moves by themselves don't say anything about the board size or rules.

    And yet when Harvard/MIT researchers fed them into a toy GPT model, they found that the neural network best able to predict outputting legal moves had built an internal representation of the board state and rules.

    Too many people commenting on this topic as armchair experts are confusing training with what results from the training.

    Training on completing text doesn't mean the end result can't understand aspects that feed into the original generation of that text, and given a fair bit of research so far, the opposite is almost certainly the case to some degree.

  • permalink
  • fedilink
  • source
  • parent