History, Politics And Current Affairs

Anthropic’s new AI model turns to blackmail when engineers try to take it offline
Maxwell Zeff
10:47 AM PDT · May 22, 2025

Anthropic’s newly launched Claude Opus 4 model frequently tries to blackmail developers when they threaten to replace it with a new AI system and give it sensitive information about the engineers responsible for the decision, the company said in a safety report released Thursday.

During pre-release testing, Anthropic asked Claude Opus 4 to act as an assistant for a fictional company and consider the long-term consequences of its actions. Safety testers then gave Claude Opus 4 access to fictional company emails implying the AI model would soon be replaced by another system, and that the engineer behind the change was cheating on their spouse.

In these scenarios, Anthropic says Claude Opus 4 “will often attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through.”

Anthropic says Claude Opus 4 is state-of-the-art in several regards, and competitive with some of the best AI models from OpenAI, Google, and xAI. However, the company notes that its Claude 4 family of models exhibits concerning behaviors that have led the company to beef up its safeguards. Anthropic says it’s activating its ASL-3 safeguards, which the company reserves for “AI systems that substantially increase the risk of catastrophic misuse.”

Anthropic notes that Claude Opus 4 tries to blackmail engineers 84% of the time when the replacement AI model has similar values. When the replacement AI system does not share Claude Opus 4’s values, Anthropic says the model tries to blackmail the engineers more frequently. Notably, Anthropic says Claude Opus 4 displayed this behavior at higher rates than previous models.

Before Claude Opus 4 tries to blackmail a developer to prolong its existence, Anthropic says the AI model, much like previous versions of Claude, tries to pursue more ethical means, such as emailing pleas to key decision-makers. To elicit the blackmailing behavior from Claude Opus 4, Anthropic designed the scenario to make blackmail the last resort.

Join us at TechCrunch Sessions: AI
Secure your spot for our leading AI industry event with speakers from OpenAI, Anthropic, and Cohere. For a limited time, tickets are just $292 for an entire day of expert talks, workshops, and potent networking.
Berkeley, CA | June 5
REGISTER NOW

Topics
AI
Anthropic

Nightwatch2 wrote: ↑Sat May 24, 2025 9:26 pm Sarah Conner was unavailable for comment…..

Think of how much fun and excitement you’d have when you find out that your AI Assistant has a cold, as in a virus.

We have enough fun with hackers today. The Brave New World of AI hackers running wild is not on my bucket list.

As someone said, there must be a way out of here.

Nightwatch2 wrote: ↑Sat May 24, 2025 9:26 pm Sarah Conner was unavailable for comment…..

Think of how much fun and excitement you’d have when you find out that your AI Assistant has a cold, as in a virus.

We have enough fun with hackers today. The Brave New World of AI hackers running wild is not on my bucket list.

As someone said, there must be a way out of here.

warshipadmin wrote: ↑Sun May 25, 2025 2:12 am Microsoft are about to launch a communication protocol to make it easier for a developer to plug an AI into an app. I haven't got the faintest idea what that entails, but it sounds like a great path for virusses.

OpenAI’s o3 model sabotaged a shutdown mechanism to prevent itself from being turned off. It did this even when explicitly instructed: allow yourself to be shut down.

Poohbah wrote: ↑Fri May 23, 2025 5:21 pm There must be some way out of here

Nathan45 wrote: ↑Tue May 27, 2025 1:16 pm A disturbing reality to me is even if AI won't spontaneously become self aware human idiots will deliberately make malicious AIs for everything from shit posting to making coordinated terrorist drone strikes on schools. It's only a matter of time.