Anthropic Discovers the Risks of What Anthropic Is Building

BREAKING NEWS. Yesterday, Anthropic published a long note that explained AI was progressing toward recursive self-improvement, which could result in humans losing control.

“If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important.”

Thank you for letting us know, Anthropic.

As usual, it’s hard to tell if the (not so) subtle point is to praise Claude’s latest features and efficiency. But somehow, it’s surprising that the company’s cycle of such "surprise announcements" comes as a surprise.

Self-improvement, replacing human cognitive capacities, efficiency over safety, etc. Was this not what most AI giants were aiming for since they started?

“What should we do?”, asks the Anthropic "Philosophy Department".

Don’t worry, they have a solution in mind: “We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.”

So, basically, stop doing what they’re doing. But wait, we should worry a little because Anthropic says: “We don’t have that long.”

In the meantime, Anthropic is planning an IPO this year, hoping to achieve a close to 1-trillion-dollar valuation, and has just raised $65 billion dollars to continue growing. Seems like a coherent way to go.

Did you find this reflection interesting?

⮕ Read more articles on LinkedIn.

⮕ Subscribe to the OFF Movement newsletter.

⮕ Request a talk or conference.