Anthropic Frontier Model Security Framework

TL;DR

Anthropic has proposed a set of cybersecurity best practices for frontier AI models, emphasizing that the strategic importance of these models requires security levels exceeding standard commercial technology. The framework centers on two-party control for critical infrastructure and the adoption of secure software development standards to protect model weights and research from theft or misuse.

Multi-Party Authorization for AI-Critical Infrastructure

Anthropic advocates for "two-party control," a security pattern where no single person has persistent access to production-critical environments. To gain access, an individual must request time-limited permission from a coworker and provide a business justification.

This approach, which Anthropic calls multi-party authorization to AI-critical infrastructure design, is intended to mitigate insider risk and defend against advanced threat actors. Anthropic notes that this pattern is already common in high-security domains such as financial services, medical device manufacturing (ISO 13485), and food safety (ISO 22000), and is applicable even for emerging AI labs with limited resources.

Secure Model Development Framework

To ensure the integrity and provenance of AI systems, Anthropic recommends the adoption of a secure model development framework based on existing software security standards. Specifically, the lab points to two gold-standard frameworks:

  • NIST Secure Software Development Framework (SSDF): A set of practices to reduce the number of vulnerabilities in released software.
  • Supply Chain Levels for Software Artifacts (SLSA): A framework for ensuring the integrity of software artifacts.

Anthropic argues that producing and deploying a model is functionally identical to building and deploying software. By implementing SSDF and SLSA, labs can establish a "chain of custody," allowing a deployed model to be tied back to the company that developed it. Anthropic encourages NIST to extend the SSDF to explicitly encompass model development within its standard-setting process.

Public-Private Cooperation and Regulatory Approach

Anthropic suggests that frontier AI research should be treated as "critical infrastructure," similar to the financial services sector. This designation would facilitate enhanced information sharing and cooperation between government agencies and AI labs to better defend against highly resourced malicious actors.

Regarding implementation, Anthropic proposes a phased approach to adoption:

  1. Voluntary Arrangements: Initial security measures can begin as voluntary agreements between labs.
  2. Procurement Requirements: Governments can use procurement guidelines (similar to EO 14028) to mandate these security standards for AI companies and cloud providers contracting with the government.
  3. Regulatory Mandates: Over time, government regulatory powers may be used to mandate compliance for the broader industry.

Implementation Status

Anthropic is currently implementing two-party controls, SSDF, and SLSA. The company acknowledges that while security measures can sometimes interfere with productivity, these precautions are necessary as model capabilities scale and will require an iterative process of enhancement in consultation with industry and government partners.

Sources

Related

  • Dispatch
    Anthropic Third-Party Testing AI Policy Proposal

    Anthropic proposes a third-party testing regime for frontier AI systems to validate safety, prevent national security risks, and avoid regulatory capture through a diverse ecosystem of auditors.

  • Dispatch
    Anthropic Increases Support for Public First Action to $40 Million

    Anthropic has donated an additional $20 million to Public First Action, bringing its total contribution to $40 million to support nonpartisan public education and policy efforts for AI safeguards.

  • Dispatch
    Anthropic Trustworthy Agents Framework and Implementation

    Anthropic outlines a multi-layered approach to building trustworthy AI agents by combining model training, system harnesses, tool permissions, and open standards to mitigate risks like prompt injection and unintended autonomy.

  • Dispatch
    Anthropic's Core Views on AI Safety

    Anthropic outlines an empirically-driven, portfolio-based approach to AI safety to mitigate catastrophic risks associated with the rapid scaling of transformative AI systems.

  • Dispatch
    Anthropic Endorsement of California SB 53

    Anthropic has endorsed California Senate Bill 53 (SB 53), a regulatory framework for powerful AI systems that prioritizes transparency and disclosure over prescriptive technical mandates.