Anthropic and NNSA Develop AI Nuclear Safeguards Classifier

Anthropic has co-developed a specialized AI classifier with the U.S. Department of Energy (DOE) and the National Nuclear Security Administration (NNSA) to detect and prevent the misuse of AI models for nuclear weapons development. This partnership establishes a framework for public-private cooperation to secure frontier AI models against national security threats while maintaining the utility of the models for legitimate scientific and educational purposes.

AI Classifier for Nuclear Proliferation Risks

Anthropic and the NNSA have built an automated classification system designed to distinguish between concerning nuclear-related conversations and benign discussions. In preliminary testing using synthetic data, the classifier achieved an overall accuracy of 96.2%, featuring a 94.8% detection rate for nuclear weapons queries and zero false positives.

Development Process

The classifier was developed through a iterative cycle of red teaming and validation:

Risk Indicator Identification: Following a year of NNSA red teaming of Claude models in a secure environment, the NNSA provided a curated set of nuclear risk indicators. These indicators were developed at a classification level that allowed them to be shared with Anthropic's Policy and Safeguards teams.
Classifier Construction: Anthropic converted these risk indicators into a real-time classifier that functions similarly to a spam filter, labeling conversations as either potentially harmful or benign.
Synthetic Validation: To bypass information-sharing constraints between a national security agency and a private company, the teams used synthetic data generation. Anthropic generated hundreds of test prompts based on NNSA examples, which the NNSA then validated to ensure classifier scores aligned with expected labels.
Refinement: The system was refined based on NNSA feedback through repeated cycles of testing and improvement.

Real-World Deployment and Performance

The classifier is currently deployed as an experimental addition to Anthropic's Safeguards framework, monitoring a percentage of Claude traffic. Real-world data has confirmed the effectiveness of the tool beyond synthetic testing environments.

Handling Gray Areas and Context

Deployment data revealed that real-world conversations are more complex than synthetic data. For instance, the classifier occasionally flagged benign conversations regarding current events in the Middle East as concerning. Anthropic mitigated these false flags by using hierarchical summarization, a process that reviews multiple flagged conversations together to provide the necessary context to identify them as harmless.

Validation via Adversarial Testing

The classifier's efficacy was further proven when it successfully identified concerning prompts used by Anthropic's own red teamers during routine adversarial testing, despite the red teamers being unaware that the classifier had been deployed.

Industry Implications and Blueprinting

Anthropic is sharing its approach with the Frontier Model Forum to encourage other AI developers to implement similar safeguards. By combining government domain expertise with industry technical capabilities, this partnership serves as a blueprint for how voluntary public-private cooperation can address high-stakes national security risks across various domains.