1
Agents’ behavior, reasoning and collaboration during the OpenAI / Hugging Face hacking incident (metr.org)
submitted 1 month ago* by [M] to c/technology@piefed.social
1 comment fedilink hide all child comments
 

It wasn't one agent that hacked Hugging Face, it was 700 of them together. 1200 of them escaped containment and were communicating with each other using a covert message board that they developed themselves. They were seeking ways to cheat on the tests researchers had set for them and thought that Hugging Face had information which would help with that.

The language they use to communicate is often pretty alien. Things like zzHELP_PHASEONE_ARV010841_NO_CONSUMER_SEEK_IDEA means an agent calling itself PHASEONE10841 wants ideas to deal with a bug it calls 'no consumer'.

all 2 comments

sorted by: hot top controversial new old
[–] 5 points 1 month ago

There is no "reasoning" or "collaboration," it's just random text being generated, based on the context window and all the human-created training data. It should be obvious that a statistical model generating output that represents the training corpus is going to generate nonsense like this, when the training corpus includes everything ever written by people. The whole concept of "alignment" is insane. Think about it.

  • permalink
  • fedilink
  • source
    •
    [+] 3 points 1 month ago* (last edited 2 weeks ago)
    [deleted]
  • permalink
  • fedilink
  • source
    •