NVIDIA/aicr

Tooling for optimized, validated, and reproducible GPU-accelerated AI runtime in Kubernetes

What it solves

Running GPU-accelerated Kubernetes clusters is often unreliable because small mismatches between kernel versions, drivers, container runtimes, and Kubernetes releases can cause difficult-to-diagnose failures. AI Cluster Runtime (AICR) eliminates this guesswork by providing version-locked, validated configurations called "recipes" that ensure the right combination of software and hardware is used for a specific workload.

How it works

AICR uses a recipe engine that matches a user's target environment (cloud provider, GPU type, OS, and workload intent) against a library of validated overlays and mixins. It then uses a bundler to convert these recipes into deployment-ready artifacts for popular GitOps tools like Helm, Argo CD, Flux, and Helmfile. To ensure the cluster is running as intended, a Snapshot Agent captures the live state of the cluster, which the AICR validator then compares against the original recipe to check for configuration drift or performance issues.

Who it’s for

Platform engineers and DevOps teams who manage GPU-accelerated Kubernetes clusters for AI training and inference workloads across various cloud providers (AWS, GCP, Azure) or on-premises environments.

Highlights

  • Multi-Deployer Support: Generates compatible bundles for Argo CD, Flux, Helm, and Helmfile.
  • Comprehensive Validation: Includes multi-phase validation for deployment, performance (training/inference), and conformance.
  • Supply Chain Security: Implements SLSA Build Level 3 provenance, signed SBOMs, and image attestations using Cosign/Sigstore.
  • Drift Detection: Ability to compare cluster snapshots to identify configuration changes over time.
  • Broad Hardware Support: Validated for a wide range of NVIDIA GPUs, including H100, A100, B200, and L40S.

Related

  • Project
    gpustack/gpustack

    An open-source GPU cluster manager that orchestrates inference engines and provisions GPU instances for scalable AI model serving.

  • Project
    raids-lab/crater

    A Kubernetes-native control plane for shared AI computing clusters that simplifies GPU resource governance, workload scheduling, and asset management.

  • Project
    GoogleCloudPlatform/cluster-toolkit

    Google Cloud Cluster Toolkit is an open‑source CLI that turns YAML blueprints into Terraform/Packer code to provision AI/ML or HPC clusters (Slurm or GKE) on Google Cloud, and also lets you submit containerised AI jobs. It provides pre‑built bundles, automatic dependency handling, and advanced features like TPU slicing and Pathways support.

  • Project
    cncf/k8s-ai-conformance

    A certification program that defines standardized capabilities for Kubernetes platforms to ensure AI and machine learning workloads are portable and reliable across different clusters.

  • Project
    k8sgpt-ai/k8sgpt

    k8sgpt is a CLI (and optional operator) that scans Kubernetes clusters, runs built‑in analyzers, and uses LLMs (OpenAI, Azure, Bedrock, etc.) to turn raw diagnostics into plain‑English explanations and remediation advice. It supports multiple back‑ends, anonymizes data, offers an MCP server for tool integration, and can be installed via Homebrew, RPM/DEB, or Windows binaries.