715
Car Wash Test on 53 leading AI models: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?" (opper.ai)
submitted 6 months ago by to c/technology@lemmy.world
327 comments fedilink hide all child comments
 

Screenshot of this question was making the rounds last week. But this article covers testing against all the well-known models out there.

Also includes outtakes on the 'reasoning' models.

you are viewing a single comment's thread
view the rest of the comments
[–] 19 points 6 months ago (2 children)

I asked my locally hosted Qwen3 14B, it thought for 5 minutes and then gave the correct answer for the correct reason (it did also mention efficiency).

Hilariously one of the suggested follow ups in Open Web UI was "What if I don't have a car - can I still wash it?"

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 5 points 6 months ago* (last edited 6 months ago)

    My locally hosted Qwen3 30b said “Walk” including this awesome line:

    Why you might hesitate (and why it’s wrong):

    • X “But it’s a car wash!” -> No, the car doesn’t need to drive there—you do.

    Note that I just asked the Ollama app, I didn’t alter or remove the default system prompt nor did I force it to answer in a specific format like in the article.

    EDIT: after playing with it a bit more, qwen3:30b sometimes gives the correct answer for the correct reasoning, but it’s pretty rare and nothing I’ve tried has made it more consistent.

  • permalink
  • fedilink
  • source
  • parent