715
Car Wash Test on 53 leading AI models: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?" (opper.ai)
submitted 6 months ago by to c/technology@lemmy.world
327 comments fedilink hide all child comments
 

Screenshot of this question was making the rounds last week. But this article covers testing against all the well-known models out there.

Also includes outtakes on the 'reasoning' models.

you are viewing a single comment's thread
view the rest of the comments
[–] 76 points 6 months ago (4 children)

They probably added a system guardrail as soon as they heard about this test. it's been going around for a while now :)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 3 points 6 months ago (1 child)

    Article mentions that Gemini 2.0 Flash Lite, Gemini 3 Flash and Gemini 3 Pro have passed the test. All these 3 also did it 10 out of 10 times without being wrong. Even Gemini 2.5 shares highest score in the category of "below 6 right answers". Guess, Gemini is the closest to "intelligence" out of a bunch.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 6 months ago

    I'm pretty sure Google's AI is fed by the same spider that goes out and finds every new or changed web page (or a variant of that).

    As soon as someone writes an article about how AI gets something wrong and provides a solution, that solution is now in the AI's training data.

    OTOH, that means it's probably also ingesting a lot of AI generated slop, which causes its own set of problems.

  • permalink
  • fedilink
  • source
  • parent