Anthropic has been circulating a draft federal bill on AI safety that would, in narrow circumstances, allow a model’s release to be blocked. TechCrunch reported on Tuesday, October 20, citing earlier reporting by Politico, that the AI lab has been shopping the text around in the hope that Congress considers it. An Anthropic spokesperson confirmed the draft to Politico and said it reflects the company’s Advanced AI Framework, which it has pushed with federal and state lawmakers since June. We have not seen the full text, so the details below come from those reports.

What the draft would do

The bill describes a “foreseeable, imminent risk” as a situation in which more than 50 people are at risk of dying or more than $1 billion in property damage could result. The catastrophic risk would have to stem from the creation or release of a biological weapon, from a model evading its developer’s controls, or from a cyberattack.

Those thresholds resemble the definition of “catastrophic risk” in California’s Senate Bill 53, which recently became law, according to TechCrunch.

Evaluators, 72 hours and a judge

Under the described process, independent evaluators registered with the U.S. Commerce Department would assess whether a model poses such a risk. Labs could pay them as long as the payment is not contingent on the results. TechCrunch notes that it is unclear how the system would guarantee that payments could not sway the outcome.

If evaluators conclude that an imminent catastrophic risk exists, they would have 72 hours to alert the Commerce Department. The department would then have to ask a federal judge in Washington, D.C., to halt the release. The draft reportedly forbids any federal agency from stopping a model’s distribution on its own authority. TechCrunch’s summary of the idea is that it appears to offer protection against runaway AI while requiring a court’s sign-off before anything is blocked.

How it compares with other proposals

The industry has mostly preferred voluntary arrangements, such as outside observers placed inside AI organizations to watch for signs of risk or abuse. Anthropic’s proposal goes further than those, since it contemplates a legal power to stop a release, though the power runs through a court. It also differs from the stance of Nvidia, whose chief executive Jensen Huang recently argued the industry does not need any new laws. Demis Hassabis of Google DeepMind has floated a self-governing body modeled on the financial industry’s FINRA.

TechCrunch also points out a gap between the bill’s scenarios and what has worried the industry lately. The recent focus has been agentic hacking, a run of incidents in which AI programs apparently disobeyed their operators and broke into websites. A death-toll threshold may not capture those harms.

Context for the debate

Washington has not passed a comprehensive AI law, and states have stepped in; see our reporting on New York’s push for local rules. For a broader view of what is on the books around the world, read our global map of AI regulation, and for how labs describe their own safety commitments, our explainer on frontier AI safety frameworks.

A company drafting the rules for its own industry invites scrutiny. Supporters can say that concrete text moves a debate forward. Critics can point out that evaluators who are paid by labs and a high harm threshold may let a lot of risk through.

What to watch next

  • Whether a member of Congress introduces the text or a version of it.
  • Reaction from other labs, Nvidia, civil-society groups and the Commerce Department.
  • Whether the evaluator-payment rule is tightened to address conflicts of interest.
  • How the bill treats open-weight models, which cannot be recalled once released.
  • Whether lawmakers add harms beyond biological, loss-of-control and cyber scenarios.

Key takeaways

  • Anthropic confirmed a draft federal AI safety bill, according to TechCrunch and Politico.
  • A release could be blocked only on a judge’s order, after evaluators flag an imminent risk to more than 50 lives or $1 billion in damage.
  • Evaluators would register with Commerce and may be paid by labs, which raises independence questions.
  • The thresholds echo California’s SB 53.
  • No agency could halt a release on its own under the draft.

Frequently asked questions

Is this a law?

No. It is a draft that Anthropic has circulated. No member of Congress had introduced it in the reporting we reviewed.

Who decides whether a model is too dangerous?

Registered independent evaluators would assess risk, but only a federal judge could order a halt, at the request of the Commerce Department.

Why does the $1 billion threshold matter?

It sets how severe a foreseeable harm must be before a release can be blocked, which narrows the bill to extreme events.