An update on independent evaluators

Anthropic provides an update on its commitment to independent, third-party assessment of its AI models and safety practices. The company has started giving a team from METR, a nonprofit that evaluates frontier AI models, employee-like access—including company hardware, office space, and internal tools—and METR will publish its risk assessment results. Anthropic requires evaluators to have expertise in frontier model capabilities, independence (no equity or financial dependence), sufficient access to training and safeguards, and freedom to publish their observations. The company explains it cannot publicly share all safety-relevant information due to malicious-actor and commercial sensitivity concerns, so it instead grants third parties deep access to close the understanding gap. Anthropic is also working with SecureBio and Accenture, is in early discussions with organizations including Hugging Face, and is developing three tiers of evaluator access with increasing information access and publication rights. The arrangement with METR cannot be vetoed by Anthropic, though narrow categories of information—such as customer data, attorney-client privileged material, and model architecture details—can be redacted, and METR may publicly characterize redactions. Anthropic frames this as a step toward robust safety oversight and encourages every frontier AI company to voluntarily embed evaluators.

(Source：Anthropic)