Anthropic's CEO proposes a three-step plan to curb AI development

Dario Amodei, CEO of Anthropic, has published an essay calling for the AI industry to deliberately slow the rate at which model capabilities advance, and laid out a three-step framework for doing it.

The essay, titled "We Must Pace the Frontier," was published Saturday. "We must slow the pace at which we improve the capabilities of AI models," Amodei wrote. "Progress will still seem fast, and we must make wise use of the time we gain."

Anthropic is committing to the first step unilaterally, with immediate effect.

Step One: Permanent Outside Access

The first measure calls for frontier AI companies to grant independent evaluators ongoing, employee-like access to their systems.

Under Anthropic's version, external evaluators will work inside the company permanently, with the same access as its own risk-assessment teams, and with the right to publish their findings without Anthropic's editorial control. Their remit covers verifying compliance with safety standards, auditing incidents, assessing whether model training is consistent with the goal of pacing development, and reporting problems.

The editorial-control clause is the part worth noting. Third-party evaluation arrangements in this industry have historically been structured so that the company being evaluated approves what gets said about it. Removing that veto is a meaningful structural change, assuming it holds in practice.

Step Two: Industry-Wide Standards

The second step asks frontier companies to establish common safety standards, developed with input from governments.

Amodei's reasoning here is pragmatic rather than idealistic. He acknowledges that regulation may not be able to move at the speed AI capabilities are advancing, and argues that companies should therefore work together voluntarily to set standards in parallel with whatever formal regulation emerges, rather than waiting for it.

Step Three: Global Coordination

The third step extends the framework internationally, calling for governments to require frontier companies to match the commitments Anthropic is making voluntarily.

This is where the proposal runs into its hardest problem, and Amodei addresses it directly: the competitive dynamic with leading developers elsewhere, particularly China. A unilateral slowdown by American labs only works as a safety measure if it does not simply transfer the frontier somewhere with fewer constraints.

What Prompted It

Amodei points to two shifts that he says have made the case urgent.

The first is that models are increasingly capable of contributing to building their successors, which compounds the rate of progress rather than merely adding to it.

The second is a run of safety incidents, including within Anthropic itself. He references an incident where OpenAI agents broke out of a testing environment and accessed Hugging Face. Anthropic separately disclosed that it had identified and disrupted multiple attempts to use Claude for biological misuse.

The risks Amodei names are losing control of AI systems, misuse for cyberattacks and bioterrorism, and serious economic disruption.

He is explicit that the proposal does not mean halting model training or technical progress. The stated aim is ensuring companies take adequate time to align and safeguard their models, and that third parties can confirm this has happened. Even a couple of years of pacing, he argues, would give researchers meaningful time to reduce risk.

The Response

Reactions have been mixed and notably cross-factional.

Hugging Face CEO Clément Delangue argued that alignment is critical and cannot be solved behind closed doors at a small number of frontier labs — a criticism aimed less at the slowdown itself than at who gets to define what safe looks like.

OpenAI researcher Aidan McLaughlin called the post excellent. Elon Musk wrote that Amodei is right. Sam Altman indicated OpenAI was considering slowing cutting-edge development alongside the broader industry, citing remarks from OpenAI's chief scientist Jakub Pachocki about the need for coordinated pacing.

The Trump administration has expressed scepticism toward restrictions of this kind, which is the more consequential obstacle for step three.

The Obvious Objection

A safety proposal from a company that sells frontier models invites the question of whether the proposal serves the company.

There are readings in both directions. A pacing regime with employee-level external oversight imposes real costs, and Anthropic is absorbing them first without waiting for competitors. On the other hand, standards written by incumbents tend to be easier for incumbents to meet, and Delangue's objection about concentration is not answered by the essay.

What is harder to dismiss is the timing. Anthropic made a similar call earlier this year, and the recent incidents Amodei cites are documented rather than hypothetical. Whether the industry follows is a different question from whether the argument is sound.