Tag: Dario Amodei

Anthropic CEO Proposes Three-Stage Plan for Coordinated AI Safety Limits

On September 12, Anthropic CEO Dario Amodei called for coordinated limits on frontier AI advancement and outlined a three-stage governance framework. The proposal begins with inviting external evaluators into the company with access comparable to internal risk teams, progresses to U.S. regulatory coordination, and ultimately seeks verifiable international agreements.

Stage One: External Evaluators With Publication Rights

Under the first stage, a proposed review team would receive company equipment, workspace access, and opportunities to speak with employees. The evaluators’ contract would permit publication of key findings without Anthropic controlling the conclusion, subject to defined legal, security, privacy, and commercial constraints. This arrangement would allow outsiders to test whether the company’s safety commitments shape real training and deployment decisions.

Amodei acknowledged that access inside one lab cannot slow a competitive field. Anthropic may open its systems to review while rival companies and governments continue to accelerate.

Stage Two: U.S. Regulatory Coordination

The second stage calls for regulation and government-mediated coordination across a critical mass of U.S. frontier developers. Amodei argues that a public-benefit charter can authorize safety-minded decisions inside Anthropic but cannot bind a competitor that rejects the same trade-off. His proposal addresses that gap with common rules rather than a transfer of company ownership.

Stage Three: International Verification

The third stage seeks verifiable agreements among states, with democracies preserving enough strategic room relative to China to pace development. The framework distinguishes between three different powers often merged in the nationalization-versus-decentralization debate: public ownership of economic gains, independent access for inspection, and legally enforceable halts on advancement speed.

Anthropic’s Existing Governance Structure

Anthropic operates as a Public Benefit Corporation under Delaware law, which requires its directors to balance stockholders’ pecuniary interests, the interests of people materially affected by the business, and its specified public benefit. Its Long-Term Benefit Trust holds board-selection powers intended to support the company’s mission. While this structure authorizes safety-minded decisions internally, it does not extend to competitors.

Ownership and Control Are Different Levers

Sanders Proposal Illustrates Partial Nationalization

A June 2026 proposal from Sen. Bernie Sanders illustrates what partial nationalization could look like. His American AI Sovereign Wealth Fund would take a 50% public stake in the largest U.S. AI companies, with an independent commission exercising the voting rights. The measure remains a proposal, not enacted law.

Public equity could redirect part of the industry’s gains and give the commission influence over company decisions. However, capability thresholds, outside verification, and enforceable stop orders would still require separate legal rules.

Legal Scholars Propose Narrow Halt Power

An August 2026 legal paper by Yonathan Arbel, Simon Goldstein, and Peter Salib separates economic claims from control over decisions that ordinary rules did not anticipate. The authors propose a narrow, discretionary, and temporary government power to halt frontier training or deployment when catastrophic risk or what they call “hard” corporate power is involved. They favor conventional regulation or taxation for monopoly, inequality, and other harms.

A halt order reaches the pacing decision more directly than public equity. The state would not need to own every model or operate every laboratory before suspending covered training or deployment. Clear statutory triggers, technical competence, independent review, and limits on discretion would be needed for that authority to claim democratic legitimacy.

Government Control Creates Concentration Risk

Moving every frontier laboratory under state ownership could place model development and the decision to stop it in the same institution. A bounded halt power leaves companies in private hands while reserving an emergency intervention for defined extreme risks.

Open-Weight Models Complicate Enforcement

Open-weight models press in the opposite direction by widening access. Researchers can inspect and adapt systems without relying on a handful of corporate gatekeepers. The U.S. National Telecommunications and Information Administration concluded in 2024 that the available evidence did not justify blanket restrictions on widely available model weights.

Frontier capability changes the enforcement problem. The European Commission requires providers of general-purpose models with systemic risk to evaluate and mitigate risks, report serious incidents, and maintain cybersecurity even when a model is open-source. The Commission warns that mitigation can become harder after an advanced model has been released openly.

Open release can expand outside scrutiny and complicate later enforcement at the same time. Replication across jurisdictions makes mitigations harder to apply consistently. Distributed auditing gives more institutions the ability to challenge a captured regulator or company; unrestricted distribution of frontier weights can weaken the control points a lawful pause would need.

The Public Brake Needs Plural Oversight

State and Supranational Models

California and the European Union demonstrate how public rules can govern privately owned developers. California’s SB 53, signed in September 2025, requires large frontier developers to publish safety frameworks, provides a channel for reporting potential critical safety incidents, and protects whistleblowers. The EU imposes risk-management duties on providers of systemic-risk models, including open models.

Amodei’s proposed evaluators would provide deeper access for testing whether comparable duties affect internal decisions. A narrow, temporary halt power would give public authorities an enforcement option when a covered system crosses a legally defined risk threshold.

Hybrid Governance Structure

In this hybrid structure, governments would set binding rules for systemically significant developers, external evaluators would verify compliance, and public authorities could pause specified training or deployment. Researchers, whistleblowers, and regulators in multiple jurisdictions would retain separate routes for contesting the evidence.

The brake would need public intervention criteria tied to demonstrated capabilities or safety failures, review outside the office invoking it, and explicit expiry and renewal rules. Evaluators would need freedom to report unfavorable findings, with redactions limited to legitimate legal, security, privacy, and narrowly tailored commercial needs. Those safeguards would reduce the chance that a temporary safety intervention becomes permanent political control over general-purpose research.

Credible Pacing Requires Common Boundaries

Private development could continue inside a common regulatory boundary for as long as frontier systems remain identifiable and enforceable control points remain available. Independent institutions would inspect compliance and expose either corporate or regulatory capture.

A public stake can redistribute AI’s wealth and boardroom influence, but ownership does not specify when training must stop. Open distribution can broaden access and scrutiny, but it cannot supply an enforceable stopping rule after frontier weights have spread.

Credible pacing therefore requires every covered frontier developer to face the same public boundary. Democratic legitimacy requires independent evaluators, researchers, whistleblowers, and regulators to inspect the evidence and contest both the line and any order to halt.