190
Open-source Deepseek R1 dethrones commercial AI, now allegedly being hit by cyberattack (www.cnbc.com)
submitted 2 years ago by to c/world@lemmy.world
50 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 7 points 2 years ago (3 children)

It still can’t count the Rs in strawberry, I’m not worried.

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 5 points 2 years ago (1 child)

    Yes it can

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (2 children)
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 3 points 2 years ago* (last edited 2 years ago)

    https://ibb.co/wVNsn5H

    https://ibb.co/HpK5G5Pp

    https://ibb.co/sp1wGMFb

    https://ibb.co/4wyKhkRH

    https://ibb.co/WpBTZPRm

    https://ibb.co/0yP73j6G

    Note that my tests were via groq and the r1 70B distilled llama variant (the 2nd smartest version afaik)

    Edit 1:

    Incidentally... I propositioned a coworker to answer the same question. This is the summarized conversation I had:

    Me: "Hey Billy, can you answer a question? in under 3 seconds answer my following question"

    Billy: "sure"

    Me: "How many As are in abracadabra 3.2.1"

    Billy: "4" (answered in less than 3 seconds)

    Me: "nope"

    I'm gonna poll the office and see how many people get it right with the same opportunity the ai had.

    Edit 2: The second coworker said "6" in about 5 seconds

    Edit 3: Third coworker said 4, in 3 seconds

    Edit 4: I asked two more people and one of them got it right... But I'm 60% sure she heard me asking the previous employee, but if she didnt we're at 1/5

    In probably done with this game for the day.

    I'm pretty flabbergasted with the results of my very unscientific experiment, but now I can say (with a mountain of anecdotal juice) that with letter counting, R1 70b is wildly faster and more accurate than humans .

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 2 years ago (1 child)
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 years ago (1 child)

    “Again” so it failed the first time. Got it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 2 years ago* (last edited 2 years ago) (1 child)

    It didn't, I just wanted a short reply. Though it failed when I asked again in the same chat. But when asked to split the word to 2 parts it became sure that the correct answer is 3.

    Here is the fun part

    Certainly! Let's split "strawberry" into two equal parts and count the 'r's in each:

    1. First half: "strawb"
      Letters: S, T, R, A, W, B
      'r's: 1 (at position 3)

    2. Second half: "erry"
      Letters: E, R, R, Y
      'r's: 2 (at positions 2 and 3)

    Now, summing them: 1 (first half) + 2 (second half) = 3 'r's.

    Wait, this contradicts my earlier correction. It seems I made another mistake. Let's carefully recount:

    • First half: "strawb"
      'r's: 1 (at position 3)

    • Second half: "erry"
      'r's: 1 (only one 'r' at position 2; the third letter is 'r' but already counted in the first half).

    Wait, no—this is getting confusing. Let's re-examine the entire word:

    Full word: S, T, R, A, W, B, E, R, R, Y
    'r's: Positions 3, 8, and 9 → 3 'r's.

    I apologize for the back-and-forth. The correct total is 3 'r's in "strawberry." Thank you for your patience as I worked through this! 😊

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (2 children)

    That’s a lot of processing just to count letters. Hopefully it can add numbers without splitting the number ;)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 2 years ago

    It’s because LLMs don’t work with letters. They work with tokens that are converted to vectors.

    They literally don’t see the word “strawberry” in order to count the letters.

    Splitting the letter probably separates them into individual tokens

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 years ago (2 children)

    Is this some meme?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 7 points 2 years ago* (2 children)

    No. It literally cannot count the number of R letters in strawberry. It says 2, there are 3. ChatGPT had this problem, but it seems it is fixed. However if you say “are you sure?” It says 2 again.

    Ask ChatGPT to make an image of a cat without a tail. Impossible. Odd, I know, but one of those weird AI issues

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 2 years ago (3 children)

    Because there aren't enough pictures of tail-less cats out there to train on.

    It's literally impossible for it to give you a cat with no tail because it can't find enough to copy and ends up regurgitating cats with tails.

    Same for a glass of water spilling over, it can't show you an overfilled glass of water because there aren't enough pictures available for it to copy.

    This is why telling a chatbot to generate a picture for you will never be a real replacement for an artist who can draw what you ask them to.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    Not really it's supposed to understand what a tail is, what a cat is, and which part of the cat is the tail. That's how the "brain" behind AI works

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -2 points 2 years ago* (2 children)

    It searches the internet for cats without tails and then generates an image from a summary of what it finds, which contains more cats with tails than without.

    That's how this Machine Learning progam works

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    That isn't at all how something like a diffusion based model works actually.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 2 years ago (1 child)

    So what training data does it use?

    They found data to train it that isn't just the open internet?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    Regardless of training data, it isn't matching to anything it's found and squigglying shit up or whatever was implied. Diffusion models are trained to iteratively convert noise into an image based on text and the current iteration's features. This is why they take multiple runs and also they do that thing where the image generation sort of transforms over multiple steps from a decreasingly undifferentiated soup of shape and color. My point was that they aren't doing some search across the web, either externally or via internal storage of scraped training data, to "match" your prompt to something. They are iterating from a start of static noise through multiple passes to a "finished" image, where each pass's transformation of the image components is a complex and dynamic probabilistic function built from, but not directly mapping to in any way we'd consider it, the training data.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 2 years ago* (1 child)

    Oh ok so training data doesn't matter?

    It can generate any requested image without ever being trained?

    Or does data not matter when it makes your agument invalid?

    Tell me how you moving the bar proves that AI is more intelligent than the sum of its parts?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 2 years ago

    Ah, you seem to be engaging in bad faith. Oh, well, hopefully those reading at least now between understanding what these models are doing and can engage in more informed and coherent discussion on the subject. Good luck or whatever to you!

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 years ago (2 children)

    It doesn't search the internet for cats, it is pre-trained on a large set of labelled images and learns how to predict images from labels. The fact that there are lots of cats (most of which have tails) and not many examples of things "with no tail" is pretty much why it doesn't work, though.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] -1 points 2 years ago (1 child)

    And where did it happen to find all those pictures of cats?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    It's not the "where" specifically I'm correcting, it's the "when." The model is trained, then the query is run against the trained model. The query doesn't involve any kind of internet search.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -2 points 2 years ago (2 children)

    And I care about "how" it works and "what" data it uses because I don't have to walk on eggshells to preserve the sanctity of an autocomplete software

    You need to curb your pathetic ego and really think hard about how feeding the open internet to an ML program with a LLM slapped onto it is actually any more useful than the sum of its parts.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] -1 points 2 years ago (2 children)

    so.... with all the supposed reasoning stuff they can do, and supposed "extrapolation of knowledge" they cannot figure out that a tail is part of a cat, and which part it is.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 5 points 2 years ago

    Non thinking prediction models can't count the r's in strawberry due to the nature of tokenization.

    However openai o1 and deep seek r1 can both reliably do it correctly

  • permalink
  • fedilink
  • source
  • parent