NVIDIA/k8s-device-plugin

NVIDIA k8s‑device‑plugin – GPU support for Kubernetes

What it is

• An official NVIDIA DaemonSet that implements the Kubernetes device plugin API for NVIDIA GPUs.
• When deployed, it automatically discovers the GPUs on each node, reports their health, and makes them available to containers via the resource type nvidia.com/gpu (and optional MIG or shared‑GPU resources).

Why it matters for AI/ML

• Most deep‑learning training and inference workloads run on NVIDIA GPUs. Kubernetes clusters need a way to expose those GPUs to pods so that frameworks like TensorFlow, PyTorch, or RAPIDS can run inside containers.
• This plugin is the bridge between the cluster scheduler and the NVIDIA driver/runtime stack, allowing you to schedule GPU‑accelerated jobs just like any other resource.

Key features (as described in the README)

Feature	What it does
GPU discovery & health	Detects the number of GPUs on each node and reports health status to the kube‑let.
MIG support	Optional strategies (none, single, mixed) to expose Multi‑Instance GPU slices as separate resources (nvidia.com/mig‑…).
Shared‑GPU access	Time‑slicing or NVIDIA MPS can be enabled via a config file, allowing oversubscription (e.g., 10 logical nvidia.com/gpu per physical GPU).
Configurable device‑list delivery	Choose how the allocated GPUs are communicated to the container runtime: environment variable, volume mounts, CDI annotations, or CDI‑CRI field.
Helm chart & ConfigMap support	Deploy the plugin with Helm, passing flags, env‑vars, or a YAML config file via a ConfigMap.
Integration with GPU Feature Discovery	Emits node labels that describe GPU capabilities (memory, compute capability, etc.) for richer scheduling.
Support for multiple runtimes	Works with Docker (deprecated), containerd, CRI‑O, and any OCI runtime that can use the NVIDIA Container Toolkit.

Typical workflow

1. Prepare nodes – Install NVIDIA driver (≈ 384.81+), nvidia‑container‑toolkit (or nvidia‑docker ≥ 2.0), and set nvidia‑container‑runtime as the default OCI runtime.
2. Deploy the plugin – Apply the static DaemonSet manifest or install via the provided Helm chart.
3. Run a GPU pod – Request nvidia.com/gpu: 1 (or a MIG slice) in the pod spec. The scheduler will place the pod on a node with a free GPU, and the runtime will inject the device files.
4. (Optional) Enable time‑slicing or MPS in a YAML config file and mount it via a ConfigMap to allow many pods to share a single GPU.

Getting started (quick‑start excerpt)

# 1. Install NVIDIA Container Toolkit (see NVIDIA docs)
# 2. Deploy the plugin (static DaemonSet example)
kubectl create -f \
  https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.17.1/deployments/static/nvidia-device-plugin.yml

# 3. Launch a test pod
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: gpu-pod
spec:
  restartPolicy: Never
  containers:
  - name: cuda-container
    image: nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda12.5.0
    resources:
      limits:
        nvidia.com/gpu: 1
  tolerations:
  - key: nvidia.com/gpu
    operator: Exists
    effect: NoSchedule
EOF

The pod will run a simple CUDA vector‑add sample and print Test PASSED.

Configuration options (high‑level)

• --mig-strategy – Controls exposure of MIG slices (none, single, mixed).
• --device-list-strategy – How GPUs are passed to the runtime (envvar (default), volume-mounts, cdi-annotations, cdi-cri).
• --device-id-strategy – Use GPU UUIDs or index numbers.
• --fail-on-init-error – Fail the plugin on init errors (default true).
• --pass-device-specs – Enables interaction with Kubernetes CPUManager (requires privileged DaemonSet).
• --config-file – Path to a YAML config that can also contain sharing settings for time‑slicing or MPS.

Limitations / caveats noted in the README

• The plugin API is still beta (as of Kubernetes v1.10).
• Health checking and GPU cleanup are not comprehensive.
• Shared‑GPU modes (time‑slicing, MPS) are mutually exclusive and apply to all GPUs on a node.
• If you forget to request a GPU, the plugin will expose all GPUs inside the container – you must always specify a limit.
• The plugin only receives official support; forks are not covered.

Where to find more

• Full documentation lives under docs/ (e.g., GPU Feature Discovery, CDI usage).
• Release notes and changelog are linked in the repository.
• Issues and contribution guidelines are provided in the Issues and Contributing section.

---

All information above is taken directly from the repository’s README; no external assumptions have been added.