Anthropic Confidential Inference via Trusted Virtual Machines

Anthropic has introduced a research framework for Confidential Inference, a system designed to provide cryptographic guarantees that sensitive user data and frontier model weights remain private. By utilizing trusted virtual machines and confidential computing, Anthropic aims to ensure that data is only decrypted within highly restricted, verifiable environments at the exact moment of processing.

Core Objectives of Confidential Inference

Confidential Inference serves two primary security purposes: protecting model weights and ensuring user data privacy.

Model Weight Security: It acts as a component in the broader effort to secure frontier models against capable threat actors.
User Security: It provides a mechanism to prove that sensitive user information is kept private throughout the inference process.

Technical Architecture: The Inference Service

Confidential Inference is built on the principle that sensitive data must remain encrypted except at the point of processing. The system employs a chain of trust to attest to software security and enforce rules regarding which software can access encryption keys.

The Role of the Trusted Loader

Because many hardware accelerators do not yet fully support confidential computing, Anthropic is exploring an implementation using a small, secure "model loader and invoker." This loader operates within a trusted environment and performs the following functions:

Decryption: It accepts encrypted data, decrypts it, and sends it to the accelerator.
Invocation: It invokes calls against the accelerator and returns the encrypted results to the caller.

In this architecture, the loader is the only component capable of accessing decrypted data. The rest of the system is considered "untrusted" and can only send requests to the loader. The loader runs on a separate virtual machine isolated by the hypervisor and presents itself as a "virtual accelerator" to the Inference Server. To maintain security, the loader only accepts programs signed by a secure continuous integration server, ensuring all executed code has undergone multi-engineer review.

The Trusted Environment and Attestation

To ensure the loader runs correctly, it operates within a confidential computing environment characterized by three key features:

Encrypted Memory: Memory is isolated by hardware from other workloads. While this protects against physical attacks and malicious hypervisors, Anthropic notes that features for sharing encrypted host memory with accelerators are not yet well established.
Disabled Debugging: Debugging features are disabled to prevent unauthorized access.
Cryptographic Proof: The system provides proof that the correct code is being executed.

These protections are managed via a Trusted Platform Module (TPM) acting as the root of trust. The TPM measures the boot process and generates a hash that serves as an attestation. A keyserver then verifies this proof—confirming the loader is isolated, running signed code, and has debugging disabled—before releasing the necessary decryption keys.

Data Flow and Lifecycle

For a user request, the data lifecycle is designed to ensure decryption only occurs in secure contexts:

Initial Encryption: The request is encrypted before it reaches Anthropic servers.
API Server Processing: The API server decrypts, processes, and re-encrypts the request before passing it forward.
Inference Server Handling: The Inference Server handles the request in encrypted form; it is only decrypted when sent to the trusted loader.
Final Delivery: Completions are encrypted before leaving the loader and are passed back through the API server to the caller.

Model weights follow a simpler path: they are stored encrypted, decrypted only at the loader, and never released from that environment.

Future Directions and Hardware Requirements

Anthropic is exploring further safeguards at the loader layer, including egress bandwidth limitations on servers holding cleartext model weights and requiring signatures from safety classifiers to run inference.

Additionally, Anthropic encourages hardware designers to incorporate confidential computing and a hardware root of trust directly into accelerators to significantly reduce the trust boundary of these systems.