NVIDIA/gpu-operator

NVIDIA GPU Operator creates, configures, and manages GPUs in Kubernetes

What it solves

Configuring Kubernetes nodes with NVIDIA GPUs is typically difficult and error-prone because it requires managing multiple software components, including drivers, container runtimes, and libraries, across different nodes.

How it works

It uses the Kubernetes operator framework to automate the deployment and lifecycle management of all necessary NVIDIA software components. These components—including NVIDIA drivers, the Kubernetes device plugin for GPUs, the NVIDIA Container Runtime, and DCGM-based monitoring—are run as containers, allowing them to be easily started, stopped, or swapped.

Who it’s for

Kubernetes cluster administrators who need to manage GPU nodes as easily as CPU nodes, particularly in environments that need to scale quickly across cloud or on-premises infrastructure using standard OS images.

Highlights

  • Automates provisioning of NVIDIA drivers, device plugins, and container runtimes.
  • Enables the use of standard OS images for both CPU and GPU nodes.
  • Simplifies scaling by automating software component lifecycles.
  • Provides automatic node labelling and DCGM-based monitoring.

Related

  • Project
    NVIDIA/k8s-device-plugin

    The NVIDIA k8s‑device‑plugin is an official DaemonSet that implements Kubernetes’ device‑plugin API for NVIDIA GPUs. It discovers GPUs, reports health, and makes them schedulable via the `nvidia.com/gpu` resource (with optional MIG and shared‑GPU modes). Install the NVIDIA driver and container toolkit on each node, deploy the plugin (static manifest or Helm), then request GPUs in pod specs. Configuration is possible via flags, env‑vars, or a YAML file, covering MIG strategy, device‑list delivery, shared‑GPU (time‑slicing or MPS), and more. The plugin is beta, lacks full health‑check/cleanup, and only the official NVIDIA version is supported.

  • Project
    NVIDIA/NVSentinel

    NVSentinel is an automated GPU fault detection and remediation system for Kubernetes that monitors hardware health and performs node cordoning, draining, and repairs to ensure reliable AI infrastructure.

  • Project
    kaito-project/kaito

    A Kubernetes operator suite that automates the deployment, scaling, and management of LLM inference, fine-tuning, and RAG engines.

  • Project
    ROCm/k8s-device-plugin

    A Kubernetes device plugin that enables the registration and scheduling of AMD GPUs as compute resources within a container cluster.

  • Project
    ray-project/kuberay

    KubeRay is a CNCF‑sandbox Kubernetes operator that provides CRDs (RayCluster, RayJob, RayService) to declaratively deploy and manage Ray clusters for AI/ML training and serving, with optional CLI plugin, API server, and dashboard.