NVIDIA/k8s-device-plugin

What it solves

This project provides a way for Kubernetes clusters to recognize and utilize NVIDIA GPUs. By default, Kubernetes does not know how to manage GPUs as first-class resources; this plugin allows the cluster to automatically detect the number of GPUs on each node, monitor their health, and allocate them to containers that request them.

How it works

It operates as a DaemonSet, meaning it runs on every node in the cluster. It implements the Kubernetes device plugin framework to expose the nvidia.com/gpu resource type. When a pod requests a GPU, the plugin coordinates with the NVIDIA Container Toolkit to ensure the hardware is correctly injected into the container.

It supports several advanced allocation strategies:

• MIG (Multi-Instance GPU): Allows partitioning a single GPU into multiple smaller instances.
• Time-Slicing: Allows multiple workloads to interleave their use of a single GPU by creating virtual replicas of the resource.
• MPS (Multi-Process Service): Provides space partitioning to explicitly limit memory and compute resources per workload.

Who it’s for

Cluster administrators and DevOps engineers who are deploying AI, machine learning, or high-performance computing (HPC) workloads on Kubernetes using NVIDIA hardware.

Highlights

• Automatic Resource Exposure: Automatically reports GPU counts to the Kubernetes API.
• Flexible Sharing: Supports CUDA Time-Slicing and MPS for oversubscribing GPUs.
• MIG Support: Enables the use of Multi-Instance GPUs for better hardware utilization.
• Customizable Allocation: Offers multiple strategies for passing device IDs (UUID vs index) and device lists (environment variables, volume mounts, or CDI annotations).
• GPU Feature Discovery: Includes implementation for automatic node labeling based on GPU features.