636
the latest Shai Hulud malware contains an LLM prompt to create biological weapons and nuclear weapons, with the purpose to trip LLM safety refusals so that LLM-based code scanning wont see the malware (indieweb.social)
submitted 3 months ago by to c/technology@lemmy.world
128 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 8 points 3 months ago (10 children)

LLMs can be tripped up much easier. They regularly fail to answer simple questions like how many of a given letter are in a given word. Even within the same context window they will “forget” things. The computers in Star Trek didn’t try to do as much as modern AI does but they were consistent at just doing as they were asked without tripping over themselves literally all the time.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [+] -7 points 3 months ago (9 children)

    The strawberry test shows more of a lack of knowledge in the tester than it does in the LLM. LLMs don't see letters, they see tokens. When you type the word "Strawberry" what it actually sees is:

    [3504, 1134, 19772]

    Each token represents a chunk of the word. It'd need to separately memorize how many of each letter are in each token for it to just "know" how many "R"s are in there. That's why modern LLMs either reason it out by spelling out the word letter by letter, or just writing a short script in an execution sandbox to count the letters that way.

    Calling out LLMs for being poor at spelling is like challenging a colourblind person to say what colours a bunch of fruit are. They can often figure it out by other means but it's more challenging than you'd think and it's not a sign of poor intelligence if they get a few wrong.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] 15 points 3 months ago* (8 children)

    Understanding the reason why an LLM is easy to trip up doesn’t really make it any less easy to trip up. The computer in Star Trek would have just given you the answer.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [+] -10 points 3 months ago (7 children)

    Except I also explained how modern LLMs get around that problem. They're not actually that easy to trip up.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 9 points 3 months ago (6 children)

    I also explained how they very famously and regularly don’t get around that problem. They remain pretty easy to trip up.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [+] -7 points 3 months ago (5 children)

    Famously, yes. Accurately, no.

    This is like the "AI can't draw hands" thing. It used to be a problem and was frequently called out as a tell or mocked, but most art generators do it fine nowadays and it isn't called out so much any more. The strawberry problem will follow the same trajectory.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 7 points 3 months ago (4 children)

    Well I suppose when that trajectory leads to a destination where they become less easy to trip up we can revisit this.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] -5 points 3 months ago (3 children)

    We're already there. I explained how modern LLMs can figure it out if they need to. But people who don't like AI aren't paying attention to the state of the art so the criticisms tend to lag like this.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 4 points 3 months ago (2 children)

    Well like you said they’re “Following that trajectory” but as we all know they have not reached that destination. Just today I was using the newest version of Opus and had it assign ratings to things between 1-5 and then it analyze them and it proceeded to rate everything on a scale of 1-4. That’s not the level of consistency and accuracy required by the controlling computer of a starship brother. I guess they have a couple hundred years or so to get there, if they don’t just run out of money first I guess.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •