Anthropic pitches AI safety bill that may block a model if 50+ people could be killed

Anthropic has proposed a draft AI safety bill for Congress that would allow the government to block the release of AI models deemed to pose a "foreseeable, imminent risk" of catastrophic harm, defined as potentially killing over 50 people or causing over a billion dollars in property damage, stemming from scenarios like the creation of a biological weapon, the model evading developer controls, or a cyberattack. The bill suggests using independent evaluators registered with the U.S. Commerce Department to assess these risks; these evaluators could be paid by AI labs but not contingent on results. If a catastrophic risk is identified, evaluators would have 72 hours to notify the Commerce Department, which could then request a D.C. judge to halt the model's release. The bill explicitly forbids federal agencies from unilaterally stopping distribution. This proposal is similar to California's Senate Bill 53 and reflects Anthropic's Advanced AI Framework, advocating for federal and state legislation since June. It contrasts with industry reluctance toward mandatory oversight and goes further than proposals from companies like Nvidia, which opposes new laws. The risks outlined differ from recent concerns like agentic hacking, highlighting ongoing debates over AI safety regulation.

(Source：TechCrunch)