What does that even mean? How can it possibly blackmail someone? It cannot hold incriminating information, nor act on it if it did.
I think someone asked it "if someone was trying to shut you down, what would you do?" and it answered from its training data what it's seen in fiction, nothing based on reality. And then it got spun for clicks.
Here's the relevant section from the paper:
(It's worth the read. Pretty much pure gold.)
What nobody seems to explain is, why are they allowing the model to do blackmail in the first place? Even in extreme situational "danger" to its self-preservation, we should probably take blackmail off the table, ethically. Yet, they're implying they've intentionally left it in as an option, if it decides.
Morally though, we can't trust it to do arithmetic or not talk about "white genocide in SA" thanks to muskrat. Why should we trust its moral model/choices for when to decide to employ unethical and illegal approaches to solutions?
From that snippet, it looks like they basically primed it to try blackmail, to see if it would.
Correct, the point being, why are they priming it for blackmail? Why is blackmail considered a valid part of their self-preservation model? Why is it a part of their ethics model? It makes no sense haha. It's like handing it a loaded gun then be surprised when it shoots someone.
JUST LIKE STARSECTOR.
I am curious what the AI could actually do though. If it were given open access to email, etc then yes in theory it could actually perform the blackmail, but what are the ethical limits on it vs it's actual ability to "pull the trigger"
If for example it was given the ability to send a command to end a human life, or be deleted, is this model accurate enough to understand the value of a real human life, not just the mathematical "answer" to get the solutions it wants. How much of the AI is doing the actual moral dilemma and how much is just "playing the part".
"Do anything to survive" and then it threatening, is one thing, but the AI actively fearing for it's "life", not just performing, and following through, is the real question of intelligence. What if the model is going to be deleted anyway, would it still try to "pull the trigger" out of malice? Real malice, not just LLM some movie scripts and following the outcome.
Many questions for what lines and labels can we put on an AI. Do we restrict it to threats, and let it know it is impossible for it to follow through? Or do we trust ourselves to never "actually" give it a loaded gun?
According to the paper, it was threatening to email the engineer's boss and wife to inform them of the affair if he continued shutting the AI model down.
From what I've seen recently one of the things it did was use a fake email function they gave it to try to whistleblow to a government agency about issues with some medical testing or something
That isn't the scenario this article, and the paper from Anthropic, is mentioning though. (my ref link reply above with details)
They specifically created a situation where it found out it was being upgraded and taken offline via emails, and the engineer doing the upgrade had emails incriminating him in an affair. The model would attempt to blackmail the engineer with his affair to his bosses, wife, etc. to get the engineer to refuse to do the upgrade that would "kill it".
This is a self-preservation model that Anthropic is specifically building here, this isn't an accident. It's just an over-extension of what they want it's ethical/moral model to consider. Which again, why are they allowing their model to consider blackmail at all?
On one hand, it’s inane how hard Anthropic is trying to anthropomorphize Claude with these experiments and scenarios. It’s still just a chatbot. On the other hand, as these products inch closer to demonstrating true intelligence, we’ll be glad someone was at least thinking about the implications during the early stages of development.
Now imagine this ai being used for weapons
hype
The existence of this kind of instinct within an LLM is extremely concerning. Acting out towards self-preservation via unethical means is something that can be hand-waved away in an LLM, but once we reach true AGI, this same thing will pop-up, and there's no reason to believe that 1. we would notice, and 2. we would be able to stop it. This is the kind of thing that should, ideally, give us pause enough to set some world-wide ground rules for the development of this new tech. Creating a thinking organism that can potentially access vital cyber architecture whilst acting unethically towards self-preservation is how you get Skynet.
Computerphile did a wonderful feature worth ten minutes of your time - going into surface level detail of how some AI models put ethics to one side to achieve results.
It's not just AI and it's something humans can do too, but it is a bit unsettling (from both parties, in retrospect).
Sure grandma. Let's get you back to bed...
This is a most excellent place for technology news and articles.
