dstackai/dstack

What it solves

dstack provides a unified control plane to manage GPU provisioning and orchestration across diverse environments, including various GPU clouds, Kubernetes, and on-premises clusters. It removes the complexity of managing different hardware providers and infrastructure setups for AI development, training, and inference.

How it works

Users set up a dstack server and a CLI to manage their compute resources. The system uses YAML configurations to define the following components:

• Fleets: For provisioning and managing clusters across clouds and on-prem.
• Dev environments: For launching environments accessible via IDEs or AI agents.
• Tasks: For running training or batch jobs on single nodes or clusters.
• Services: For deploying model inference as scalable endpoints.
• Volumes: For managing data persistence via instance and network volumes.
• Presets: Experimental agent-driven optimization for inference.

These configurations are applied via the CLI, an API, or AI agent skills, and dstack automatically handles the underlying infrastructure tasks like auto-scaling, networking, port-forwarding, and error recovery (e.g., out-of-capacity errors).

Who it’s for

AI developers and ML engineers who need to orchestrate compute resources across multiple GPU clouds or on-prem clusters for training and deploying models.

Highlights

• Broad Hardware Support: Works out of the box with NVIDIA, AMD, Google TPU, and Tenstorrent accelerators.
• Agent-Ready: Includes specific "skills" that allow AI agents (like Claude or Cursor) to manage fleets and submit workloads.
• Unified Control Plane: Single interface for managing development, training, and inference across hybrid cloud/on-prem setups.
• Automated Infrastructure: Handles job queuing, auto-scaling, and run failures automatically.