agent-substrate/substrate

Agent Substrate – A high‑density sandbox runtime for AI agents

What it is – Agent Substrate is a Kubernetes‑native execution platform that runs actors (stateful processes such as AI agents, tool‑calling services, or any long‑running workload) inside isolated sandboxes. It multiplexes many actors onto a small pool of physical pods, achieving 10× higher density than ordinary containers while keeping a sub‑500 ms “resume” latency.

Why it matters for AI – Modern autonomous agents spend most of their time idle, waiting for a new request. Substrate exploits this by suspending an actor’s whole process (including RAM and filesystem) and restoring it on demand, letting a single worker pod host dozens of agents. This dramatically reduces the compute cost of large‑scale agent fleets and makes it feasible to run thousands of sandboxed agents on a modest Kubernetes cluster.

Core concepts

Term	Meaning
Actor	An individual agent or service that has its own state (memory, files) and can be started, stopped, or migrated.
WorkerPool	A set of Kubernetes pods that actually run the sandboxes. Workers are shared among many actors.
Atespace	A namespace‑like isolation boundary for a group of actors.
ActorTemplate	The container image and configuration that defines how a particular kind of actor should be launched.
Suspend/Resume (Teleport)	Snapshotting an actor’s full state, storing it (e.g., in PostgreSQL), and later restoring it on any free worker.

Key features (as described in the README)

Massive multiplexing – > 30× oversubscription; the demo shows ~250 stateful actors on only 8 pods.
Sub‑second activation – Resume latency under 500 ms, > 500 suspend/resume cycles per second.
Zero‑trust isolation – Supports gVisor micro‑VMs and cloud‑hypervisor microVMs for kernel‑level sandboxing.
State persistence – Full RAM and filesystem snapshots survive hibernation, enabling “actor teleport”.
Framework‑agnostic – Works with any agent stack (ADK, LangChain, Claude Code, Model Context Protocol, etc.) because it runs standard OCI containers.
Kubernetes integration – Uses native Pods, Pod autoscaling, and custom controllers (WorkerPool CRD) for scheduling and scaling.
CLI (kubectl‑ate) – Provides simple commands to create, list, suspend, resume, and delete actors.
Observability & security – Built‑in logging, metrics, JWT‑based authentication, and threat‑model documentation.

Typical use cases

Large‑scale autonomous‑agent fleets – Deploy thousands of chat‑or‑tool‑calling agents without a one‑to‑one pod per agent.
Stateful tool services – Run sandboxed code‑execution environments (e.g., Claude Code, Antigravity) that need to keep a filesystem between calls.
Research on RL‑based agent loops – Combine inference, tool use, and training cycles in a single Kubernetes cluster, letting the scheduler move actors between workers as load changes.
Secure multi‑tenant execution – Use gVisor or micro‑VM isolation to protect each agent’s code and data.

Project status

Early development – APIs are still evolving; not production‑ready.
Supported Kubernetes versions – Latest stable release and the previous minor release.
Roadmap – Detailed in docs/roadmap.md; includes stability, richer autoscaling, and broader sandbox support.

Quick‑start (development) summary

# Prereqs: go, kubectl, docker (kind will be installed automatically)
hack/create-kind-cluster.sh                     # spin up a local kind cluster
hack/install-ate-kind.sh --deploy-ate-system    # install Substrate control plane
hack/install-ate-kind.sh --deploy-demo-counter   # deploy the sample counter actor
go install ./cmd/kubectl-ate                    # install the CLI
kubectl ate create actor my-counter-1 -a ate-demo-counter --template counter
kubectl port-forward -n ate-system svc/atenet-router 8000:80
curl -X POST -H "ate-target-actor: ate-demo-counter/my-counter-1" http://localhost:8000/

The same flow works on GKE using the setup-gcp tooling described in the README.

Ecosystem & integrations

Agent Executor (google/ax) – A distributed agent runtime built on Substrate.
kagent – CNCF‑sandbox project that uses Substrate to run stateful AI agents.
Demo collection – Counter, sandbox, Claude Code multiplex, request parking, autoscaled worker‑pools, etc.

Community & contribution

Google Group: ate-dev for announcements and discussion.
Weekly sync: Thursdays 10 am – 11 am PST (video call & dial‑in).
Slack: CNCF channels #substrate-users and #substrate-dev.
Contributing – See CONTRIBUTING.md; the project is very young, so contributions should align with core system and demo work.

Where to learn more

Architecture doc – docs/architecture.md
API guide – docs/api-guide.md
CLI docs – cmd/kubectl-ate/README.md
YouTube channel – https://www.youtube.com/channel/UCN9PPqlTtVxlcpbQ-NWpfZQ (demo videos)

Bottom line – Agent Substrate is a purpose‑built, Kubernetes‑based runtime that lets you run many isolated, stateful AI agents (or any similar workload) on a small cluster with fast suspend/resume and strong security guarantees. It is still experimental but provides a concrete foundation for building large‑scale, cost‑effective agent infrastructures.