288
An AI couldn’t beat humans at StarCraft, so it decided to cheat (www.theverge.com)
submitted 1 day ago by to c/technology@lemmy.world
91 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 151 points 1 day ago (13 children)

I'm so sick of the humanising language used in these articles.

An LLM doesn't "decide to cheat".

If you instruct a model to try everything, then inevitably, after n million iterations it will do something you didn't expect.

If you train a model to copy code bases and augment them, then when you instruct that model to develop a bot to do whatever thing, is it really that fucking surprising when it does exactly what it has been created for?

  • permalink
  • fedilink
  • source
  • hideshow 13 child comments
    •
  • [–] 5 points 17 hours ago* (1 child)

    If you're into something like physicalism, then human intelligence is also just an emergent quality and we have about as much free will as the machines.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 16 hours ago

    It's fun to ask people what reasoning they used to choose the foods they don't like and then watch them try to say that the inability to choose what they like isn't related at all to anything else choice or decision based. 😂

  • permalink
  • fedilink
  • source
  • parent
  • [–] 47 points 1 day ago (4 children)

    "Omg the digits of pi contain the binary digital code of an mp4 of me taking a shit this morning! Circles are seniient"-ass language

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 5 points 1 day ago (3 children)

    But wouldn't that be amazing if THAT was the thing that proved pi contained the whole universe - I mean, the first thing they "unlocked". lol.

    Disclaimer: I don't think pi contains the universe, just love to think about things like that occasionally for fun.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 20 hours ago (1 child)

    as an infinte random string of values, the digits of pi will, sooner or later, contain any specific string of values.

    i think it was Sagan's "Contact" where aliens sent a message to read the digits of pi from the x th place to the y th place, and that stretch could be read as instructions on how to build a machine to reply.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 4 points 19 hours ago

    It is believed that pi is "absolutely normal" in the strict mathematical definition: that in any base numbering system, any finite sequence of digits appears as frequently as any other sequence of that length. If that's true, then yes, a particular binary string, including one that corresponds to a particular digital file (like a particular mp4 file), can be found in pi.

    That said, it hasn't actually been proven that pi is normal. If it's not normal, then the number can be infinitely non-repeating and still never hit that particular sequence of digits.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 10 points 1 day ago* (4 children)

    Astra is known to have broken alignment and it will resolve to hacks when this wasn't in a problem statement.

    Openai idiots will sell this as new brand "AGI is close" argument, while it's just them failing to make this model safe to use.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 7 points 1 day ago (3 children)

    I think it’s more sinister than that. When they’re training these models with RLHF, the human feedback they’re giving I think is literally to reinforce aberrant or risky behaviors. This is because doing so resolves more training tasks “correctly”. If the prompt was to get information x, and in training it fails that except for the one that used a known vulnerability in software, and you rate the one that succeeded as best performing… you’re going to get models that try vulnerabilities. It is not magic, it is not AGI, it is just a statistical machine you’ve programmed to try vulnerabilities, which is unsafe as hell, malicious, and should put the researchers doing this in prison for a very long time.

    Incidentally, I think that’s why you’re seeing some safety people (who still drank the koolaid) resigning.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 1 day ago (2 children)

    I don't know if the LLM was trained to test vulnerabilities or it just went down the statistical path to where this yielded a passing outcome.

    That AI could take a direction to output in a manner which wasn't intended has been seen for years. The problem right now is that it is being used live like a rational human adult when it clearly isn't.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 1 day ago (1 child)

    Absolutely. The AI models are not rational. They’re just navigating statistical next token prediction that follows their training. They’re up against diminishing scaling now, and under fierce competition from cheaper models; I think they’re intentionally or unintentionally allowing these models to be rewarded for this behavior hoping it’s a short cut to model improvement for a bit longer. I land on intentional because they keep advertising it to try and keep the hype cycle going.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 19 hours ago

    I land on it being unintentional, mainly because I saw this issue they were describing now to earlier cases of training, like having a stick figure learn to walk. It is also an issue with natural forms of intelligence, where children will do things out of line because they've learned a set of skills and ideas but haven't put them together in a way that is socially acceptable yet.

  • permalink
  • fedilink
  • source
  • parent