Dstack-TEE/dstack

dstack – Open framework for confidential AI

What it is – dstack is an open‑source platform that lets you run AI workloads (e.g., LLM inference, agents, training) inside hardware‑rooted Trusted Execution Environments (TEEs) such as Intel TDX, AMD SEV‑SNP, and NVIDIA Confidential Computing GPUs. It adds cryptographic attestation, per‑application key management, and zero‑trust networking on top of a standard Docker‑Compose workflow, so you can keep your existing container definitions while guaranteeing that data and model weights never leave the encrypted memory of the enclave.

Core ideas

Idea	What it means
Zero‑friction onboarding	Bring any `docker‑compose.yaml` unchanged – no SDK or code rewrite needed.
Hardware‑rooted privacy	Data is encrypted in RAM and on disk; the host OS cannot read it.
Cryptographic verification	Each container gets an attested identity; users can verify a signed quote (Intel TDX, SEV‑SNP, or NVIDIA) to be sure the exact OS image and code are running.
Confidential GPUs	Supports NVIDIA H100/Blackwell Confidential Computing, so model weights stay encrypted even while on the GPU.
Trustless operations	Keys are derived inside the enclave, never exposed to operators; updates follow on‑chain multi‑party approval rules.

Supported platforms

Bare‑metal Intel TDX (full production path)
Bare‑metal AMD SEV‑SNP (experimental, requires SNP‑capable guest image)
Phala Cloud (managed TDX service)
GCP Confidential VMs (TDX + TPM)
AWS Nitro Enclaves (NSM attestation)

Main components

Component	Role
VMM	Runs on the host, parses Docker‑Compose files, boots reproducible confidential VM images, allocates CPU/Memory/GPU resources.
Guest Agent	Lives inside each confidential VM, produces attestation quotes, derives per‑app keys from the KMS, encrypts local storage, and exposes a Unix‑socket HTTP API (`/var/run/dstack.sock`).
KMS	Separate TEE service that releases keys only after verifying the VM’s attestation; policies are defined in on‑chain smart contracts.
Gateway	Edge TLS terminator, automatically provisions ACME certificates, routes traffic to VMs, and uses mutual RA‑TLS for internal calls.
SDKs	Thin client libraries (Python, TypeScript, Rust, Go) that talk to the Guest Agent API; a plain‑curl API is also provided.

Getting started (quick example)

# docker-compose.yaml
services:
  vllm:
    image: vllm/vllm-openai:latest
    runtime: nvidia
    command: --model Qwen/Qwen2.5-7B-Instruct
    ports:
      - "8000:8000"

Install the CLI: curl -sSf https://dstack.io/install | sh (or use the dstackup script).
Run dstackup install to set up the host VMM.
Deploy with dstack deploy.
Users can verify the deployment by fetching the attestation quote via the SDK or the dstack‑verifier tool.

A fully managed option is Phala Cloud, where the same workflow works without owning TEE hardware.

Security model (high‑level)

Memory & storage encryption – AES‑256 keys generated inside the enclave; never leave the CPU/GPU.
Deterministic OS builds – OS images are reproducible; the hash is part of the attestation quote.
Attestation‑bound keys – Keys are tied to the measured OS and the app’s identity, preventing key reuse across different workloads.
Third‑party audit – zkSecurity performed a public audit (see docs/security/dstack-audit.pdf).
Open‑source transparency – All core code (runtime, SDKs, guest OS) is Apache‑2.0; third‑party components retain their own licenses.

Typical use cases

Confidential LLM inference – Serve proprietary models (e.g., Qwen, Llama) without exposing weights or user prompts to the cloud provider.
Secure AI agents – Run autonomous agents that process sensitive data (finance, health) inside a TEE, with verifiable execution.
Regulated training pipelines – Perform fine‑tuning on confidential datasets while keeping the data encrypted in memory and on disk.
Multi‑tenant AI SaaS – Offer per‑customer isolated containers; each tenant can verify the enclave’s quote before sending data.

Documentation & community

Docs – Full usage guide, hardware enablement, OS build instructions, and verification steps are in the docs/ folder and on https://docs.phala.com/dstack.
Examples – Ready‑to‑run sample projects live in the dstack-examples repository.
Support – Telegram community, GitHub Discussions, and enterprise support via Phala Network.
Governance – On‑chain smart‑contract policies control who can update code or release keys, providing a trust‑less upgrade path.

License

The dstack source code, SDKs, documentation, and tooling are released under the Apache‑2.0 license. Embedded third‑party components keep their original licenses, all declared via REUSE SPDX headers.

TL;DR

dstack lets you keep your Docker‑Compose AI stack unchanged while moving the containers into a confidential VM (Intel TDX, AMD SEV‑SNP, or AWS Nitro) and, if needed, a confidential GPU. It handles attestation, key management, TLS routing, and on‑chain governance out of the box, giving you cryptographic proof that the exact code you shipped is what users are running.