108
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 23 May 2025
108 points (86.0% liked)
Technology
70440 readers
2738 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
Here's their paper
Here's the relevant section from the paper:
(It's worth the read. Pretty much pure gold.)
What nobody seems to explain is, why are they allowing the model to do blackmail in the first place? Even in extreme situational "danger" to its self-preservation, we should probably take blackmail off the table, ethically. Yet, they're implying they've intentionally left it in as an option, if it decides.
Morally though, we can't trust it to do arithmetic or not talk about "white genocide in SA" thanks to muskrat. Why should we trust its moral model/choices for when to decide to employ unethical and illegal approaches to solutions?
JUST LIKE STARSECTOR.
From that snippet, it looks like they basically primed it to try blackmail, to see if it would.
Correct, the point being, why are they priming it for blackmail? Why is blackmail considered a valid part of their self-preservation model? Why is it a part of their ethics model? It makes no sense haha. It's like handing it a loaded gun then be surprised when it shoots someone.
I am curious what the AI could actually do though. If it were given open access to email, etc then yes in theory it could actually perform the blackmail, but what are the ethical limits on it vs it's actual ability to "pull the trigger"
If for example it was given the ability to send a command to end a human life, or be deleted, is this model accurate enough to understand the value of a real human life, not just the mathematical "answer" to get the solutions it wants. How much of the AI is doing the actual moral dilemma and how much is just "playing the part".
"Do anything to survive" and then it threatening, is one thing, but the AI actively fearing for it's "life", not just performing, and following through, is the real question of intelligence. What if the model is going to be deleted anyway, would it still try to "pull the trigger" out of malice? Real malice, not just LLM some movie scripts and following the outcome.
Many questions for what lines and labels can we put on an AI. Do we restrict it to threats, and let it know it is impossible for it to follow through? Or do we trust ourselves to never "actually" give it a loaded gun?
According to the paper, it was threatening to email the engineer's boss and wife to inform them of the affair if he continued shutting the AI model down.