you are viewing a single comment's thread
view the rest of the comments
[–] 36 points 3 months ago (12 children)

Whenever I see these, I try them out. Sometimes I can reproduce them, this one I can't. However, I remain extremely skeptical and believe that whenever one of these screenshots goes around, someone at LLM Company hard codes a fix for that specific fuck up. Against how many of these hard fixes does each LLM answer get checked nowadays?!

  • source
  • hideshow 12 child comments
  • [–] 33 points 3 months ago (3 children)

    On the very long list of shitty things about AI is the fact that they are non-deterministic.

    I was however able to get this fuckup on first try:

  • source
  • parent
  • hideshow 3 child comments
  • [–] 21 points 3 months ago (4 children)

    I just tried this one and got:

    There are 2 t's in the word colonialism:

    colonialism

    They are located at the end of the word: the t and i at the end of the sequence.

    They usually include some randomness in the responses which is why they don’t always respond the same way (and sometimes will even often but not always get the answer correct, or visa versa).

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 3 months ago (3 children)

    I am also trying from a non English speaking country, pretty sure that has an impact as well on behavior.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 3 months ago (2 children)

    That could be, but I think a lot of the models are the same because they can speak many languages pretty well. I’ve had a lot of success with talking to models I normally prompt in English using other languages. But I have also seen Chinese appear unprompted in response to English queries from certain Chinese models a few times.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 5 points 3 months ago (1 child)

    Oh yeah, an LLM itself doesn't make that much of a difference. But Google does. Features and "features" often show up later on non English Google. The AI summary took a few months to appear on German google – even if you set your language to English.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 3 months ago

    I think that's because they have to wait for more testing to make sure the model/feature is "safe" for that region and take into consideration the linguistic and cultural differences. I believe it's mostly legal stuff they need to make sure they won't be breaking laws in different countries and see if they would face backlash from authorities or the public. But I think it's mostly the same model with the same data everywhere.

  • source
  • parent
  • [–] 5 points 3 months ago* (last edited 3 months ago) (1 child)

    I tried it, and got similiar shorter answers but not the exact same answer. Sometimes it ends up getting it right at the end after fumbling a lot, and sometimes it just fails completely.

    Searching on Google directly sometimes doesn't produce the AI Overview on stuff like these in my experience, but passing the search to Google from DDG with the bang (!g) almost always produces the AI Overview.

    edit: I tried it again and it grew the ability of humor:

    There are 2 't's in the word colonialism. colt-a-ca-l-i-s-m (just kidding) C-o-l-o-n-i-a-l-i-s-m:

    • t = 0 (If you were thinking of colonization, there is still only 1 't' in the word.)

    Interestingly, on one of my attempts it used python to count the number of t's and still ended up getting the "verbal" explanation wrong.

  • source
  • parent
  • hideshow 1 child comment