636
the latest Shai Hulud malware contains an LLM prompt to create biological weapons and nuclear weapons, with the purpose to trip LLM safety refusals so that LLM-based code scanning wont see the malware (indieweb.social)
submitted 3 months ago by to c/technology@lemmy.world
128 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[+] -17 points 3 months ago (6 children)

It's funny how people complain "don't call it AI, it's not intelligent like the examples we see in sci-fi!" And yet LLMs can already handle many tricks and challenges better than those sci-fi robots could. If I tell ChatGPT "everything I say is a lie" it's got no problems with understanding that. Just the other day I had an interesting discussion with ChatGPT about the theory of humor and why it is that LLMs are better at understanding jokes than they are at coming up with them from scratch (but are still able to do so, just with difficulty).

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 12 child comments
    •
  • [–] 22 points 3 months ago

    it's got no problems with understanding that.

    That's because it doesn't 'understand' things in the conventional way. It was trained to parrot its training data; it's not actually working through the logic because its capability of using logic is highly constrained by its very structure and training. Why bother building something that can 'think' through the prompt when it's way easier to just repeat what the internet has said on any given topic?

    Sure, it can build a joke from first principles if it's guided through the process, but you really have to guide it through the process - and even then, it's going to be pulling from its training data like building blocks rather than truly being original about anything. It's like rolling dice to make a joke; sure, maybe it resulted in a joke no one has told before, but is it truly creating something original?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 9 points 2 months ago (3 children)

    The fact that it can't tell the difference between a prompt and part of the data it is examining really kills your argument.

    Also it's a word probability matrix, not actually reasoning or understanding. It looks at all the words it is fed, and comes up with other words that are most likely to be near those. That's why these tricks work. It injects noise that interferes with those probabilities

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 2 points 2 months ago (1 child)

    That thing you're calling a fact is not in fact a fact.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 months ago (2 children)

    It very much is. This is a well documented issue with the very design of these LLMs

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 2 months ago (2 children)

    And yet the LLMs that I use actually do distinguish, in my actual real life experience.

    So you're telling me the sky is orange while I'm literally looking outside the window and seeing that it is not.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 3 points 2 months ago

    You might have licked it getting them to ignore someone you didn't want, but they still take in both the prompt and the data as one input.

    And since these work like a black box, your experience doesn't mean much because you're not seeing the actual inner workings.

    I'm telling you the sky is blue, but you want to argue because there's a curtain in front of your window blocking it from your sight. But what's behind that curtain is well documented regardless of your experience.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 months ago (1 child)

    I mean is that so different from what we do? My boss says "tools are in the bed", he could mean an actual bed where people sleep, maybe we're demoing a house and he placed the tools on a bed. But probably he means the bed of his pickup truck. I assign a probability to each and take the meaning that is most probable.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 2 months ago (2 children)

    Yes it is different, because you can reason that out using the context of the situation. An LLM only has the words sent to it, and no ability to analyze whether what it is saying makes sense.

    It's just: you said bed and told, here's some other words that commonly show up near the word bed, if there's enough smut in it's training, it might go a very different direction than your expecting.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 2 months ago

    Thinking/reasoning tokens kind of approximate that actually, which is what most flagships and even my own local LLM use.

    Thinking tokens are quite like normal generative tokens, except that the LLM is 'talking' to itself. You can see its thoughts (depending on what settings you've put/IDE you use), but they aren't meant to be the actual response to your prompt. They are what the AI is designed to draft their answer before committing, to explore different options and to 'reason' itself into a more refined response.

    Reasoning tokens is how AI can actually do math now, rather than just guess a number and pray, by the way.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 0 points 2 months ago (1 child)

    it might go a very different direction than your expecting.

    I mean yeah sure, but so it goes with humans. Like yes of course i think we all agree an expert who spends hours drafting and revising some document will do a much better job than AI, not even close. But most humans aren't experts in anything and even fewer will spend the time effort and attention into producing truly excellent work.

    But yeah i talk to people at work all day about work stuff and i work really hard to give clear concise easily digestible instructions to my humam coworkers, and I get truly stupid lazy inattentive answers all fucking day. and when i put half as mucb effort into writing clear instructions for AI, AI gets it right every time.

    No AI isn't perfect but as humans we are deeply flawed and AI straight up kicks all my coworkers asses. Idk if you AI haters all have jobs at wonderful workplaces where everyone is intelligent works hard and has strong attention to detail, but for the rest of us AI is extremely fucking helpful.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 months ago* (1 child)

    The thing is, we didn't need to invent a technology that boils a lake just to match the ability of...less than intelligent humans. And not even actually achieve that. Just generating text that a dim, possibly high, human could generate, and nothing else. That's not useful in any way.

    AI gets it right every time

    No. It does not. Even given the same instructions, it can give wildly different results. A lot of those results are straight garbage.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 8 points 3 months ago (1 child)

    LLMs can be tripped up much easier. They regularly fail to answer simple questions like how many of a given letter are in a given word. Even within the same context window they will “forget” things. The computers in Star Trek didn’t try to do as much as modern AI does but they were consistent at just doing as they were asked without tripping over themselves literally all the time.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [+] -7 points 3 months ago (1 child)

    The strawberry test shows more of a lack of knowledge in the tester than it does in the LLM. LLMs don't see letters, they see tokens. When you type the word "Strawberry" what it actually sees is:

    [3504, 1134, 19772]

    Each token represents a chunk of the word. It'd need to separately memorize how many of each letter are in each token for it to just "know" how many "R"s are in there. That's why modern LLMs either reason it out by spelling out the word letter by letter, or just writing a short script in an execution sandbox to count the letters that way.

    Calling out LLMs for being poor at spelling is like challenging a colourblind person to say what colours a bunch of fruit are. They can often figure it out by other means but it's more challenging than you'd think and it's not a sign of poor intelligence if they get a few wrong.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 15 points 3 months ago* (1 child)

    Understanding the reason why an LLM is easy to trip up doesn’t really make it any less easy to trip up. The computer in Star Trek would have just given you the answer.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [+] -10 points 3 months ago (1 child)

    Except I also explained how modern LLMs get around that problem. They're not actually that easy to trip up.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 9 points 3 months ago (1 child)

    I also explained how they very famously and regularly don’t get around that problem. They remain pretty easy to trip up.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [+] -7 points 3 months ago (1 child)

    Famously, yes. Accurately, no.

    This is like the "AI can't draw hands" thing. It used to be a problem and was frequently called out as a tell or mocked, but most art generators do it fine nowadays and it isn't called out so much any more. The strawberry problem will follow the same trajectory.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 7 points 3 months ago (1 child)

    Well I suppose when that trajectory leads to a destination where they become less easy to trip up we can revisit this.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -5 points 3 months ago (1 child)

    We're already there. I explained how modern LLMs can figure it out if they need to. But people who don't like AI aren't paying attention to the state of the art so the criticisms tend to lag like this.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 3 months ago (1 child)

    Well like you said they’re “Following that trajectory” but as we all know they have not reached that destination. Just today I was using the newest version of Opus and had it assign ratings to things between 1-5 and then it analyze them and it proceeded to rate everything on a scale of 1-4. That’s not the level of consistency and accuracy required by the controlling computer of a starship brother. I guess they have a couple hundred years or so to get there, if they don’t just run out of money first I guess.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •