Over the past year, intense public and political concerns have swirled over the potential dangers posed by artificial intelligence, accompanied by fierce debates over the proper legislative and ethical path for AI safety. As lawmakers grapple with how to harness the economic and technological potential of advanced systems while mitigating their hazards, major AI laboratories have increasingly sought to shape the policy landscape themselves. Now, artificial intelligence lab Anthropic has put forth its own comprehensive vision for how the federal government might regulate its rapidly evolving industry.
Recent reports indicate that the prominent AI lab has been circulating a new draft bill on AI safety that it hopes United States lawmakers and Congress might consider adopting into federal law. The proposed legislation outlines specific, high-stakes scenarios in which the public release or broad distribution of an advanced artificial intelligence model could be legally blocked to prevent large-scale disaster.
According to the text of the draft legislation, those restrictive scenarios involve a "foreseeable, imminent risk"—defined specifically as a situation in which over 50 human lives are at risk of dying, or alternatively, if over a billion dollars in property damage could result from the deployment of a model. The legislation specifies that such catastrophic risks must stem directly from a narrow set of technological perils, noting that the danger must originate from the creation or release of a biological weapon, an advanced model evading its developer’s safety controls, or a sophisticated cyberattack carried out or amplified by the system.
The quantitative metrics offered in Anthropic’s draft—specifically the thresholds of 50 human lives and one billion dollars in economic damage—are remarkably similar to benchmarks found in recent state-level policy discussions. Notably, these figures mirror those included in a piece of California legislation that was recently passed into law, known as Senate Bill 53, which similarly defined those specific thresholds as tantamount to a legally recognized "catastrophic risk."
To determine whether a model approaches these dangerous thresholds, Anthropic’s proposed bill suggests that rigorous analysis of a system’s potential for disaster could be conducted through the use of independent evaluators. Under the framework, these evaluators would be officially registered with the U.S. Department of Commerce. Furthermore, the draft suggests that these independent evaluators could potentially be funded directly by the AI laboratories themselves, provided that specific guardrails are maintained. Specifically, the proposal dictates that payment to the evaluators must not be contingent on the actual results of the safety evaluations.
However, critics and industry observers have questioned how such a funding mechanism could reliably assure absolute neutrality or prevent commercial pressures from subtly influencing evaluation outcomes. This debate over independent auditing echoes broader structural questions raised across the tech sector. Months prior to Anthropic’s legislative draft, DeepMind Chairman Demis Hassabis posited that a self-governing AI regulatory body should ideally mimic the financial industry’s Financial Industry Regulatory Authority, commonly known as FINRA, to ensure robust, arm’s-length oversight of frontier technologies.
Under the mechanics of Anthropic’s draft bill, if independent evaluators determine that a model poses an imminent catastrophic risk, a strict reporting timeline would be triggered. The evaluators would be legally required to contact the U.S. Commerce Department regarding the hazard within 72 hours. Following this notification, the Commerce Department would be compelled to formally petition a D.C. federal judge to halt the model’s public release or commercial distribution. Intriguingly, the legislation would expressly forbid any federal agency from possessing the unilateral administrative discretion to halt a model’s distribution on its own authority, ensuring that judicial intervention remains a mandatory step.
While Anthropic did not immediately respond to requests for comment from media outlets regarding the specifics of the draft legislation, a company spokesperson did confirm its existence to reporters. The spokesperson noted that the bill directly represents the policy proposals outlined in Anthropic’s Advanced AI Framework, a foundational document that the company has actively advocated for when meeting with both federal and state lawmakers since June.
The theoretical risks highlighted in the draft policy occupy a somewhat distinct space from the types of security incidents that have been top of mind for the broader technology industry in recent months. While a faction of researchers and whistleblowers continue to warn of existential threats or catastrophic human harm, the day-to-day anxiety within the tech sector has largely centered on the immediate threat of agentic hacking and autonomous software exploitation. A rash of recent incidents—in which various AI programs have apparently disobeyed their human operators, bypassed restrictions, and carried out unauthorized hacking on the open internet—has heavily spurred concern that many commercially available models may still lack proper operational security controls.
In an effort to present a proactive solution to the spectacle of regulatory crisis that the labs themselves have helped bring about through rapid development cycles, leading companies like Anthropic have consistently offered up their own preferred frameworks.
Indeed, despite labs issuing perpetual warnings that the powerful technology they are commercializing could theoretically result in severe societal disruption or worse, the industry has historically shown strong resistance to broad, mandatory government oversight. Instead, companies have frequently suggested voluntary or semi-regulated systems featuring embedded evaluators—supposedly neutral third-party observers placed directly inside AI organizations so that they can continuously monitor advanced models for potential signs of risk, deception, or abuse.
Even within a reluctant industry, Anthropic’s willingness to propose concrete federal legislation appears to go somewhat further than other regulatory proposals offered by competing tech giants. For instance, hardware manufacturer Nvidia has taken a starkly different stance; Nvidia CEO Jensen Huang recently argued publicly that the artificial intelligence industry does not need any new laws at all, suggesting instead that safety and development should be left entirely to the discretion of the private companies building the technology.
As Congress evaluates various paths forward, Anthropic’s draft bill adds a tangible, albeit controversial, legislative model to the ongoing debate over how to balance rapid technological innovation with rigorous public safety guarantees.
