NVIDIA/DCGM

NVIDIA Data Center GPU Manager (DCGM) is a project for gathering telemetry and measuring the health of NVIDIA GPUs

What it solves

NVIDIA Data Center GPU Manager (DCGM) addresses the complexity of managing and monitoring NVIDIA GPUs in large-scale cluster environments. It provides a centralized way to handle health monitoring, diagnostics, and resource governance to ensure high reliability and uptime for data center infrastructure.

How it works

DCGM operates as a daemon that collects telemetry and manages GPU state. It provides a suite of tools including active health monitoring and comprehensive diagnostics. It can be used as a standalone tool for infrastructure teams or integrated into larger cluster management and resource scheduling systems. It also integrates with the Kubernetes ecosystem via the dcgm-exporter to gather GPU telemetry.

Who it’s for

Infrastructure teams and system administrators responsible for managing NVIDIA GPU clusters in data centers, as well as developers building cluster management tools.

Highlights

  • Comprehensive Monitoring: Includes active health monitoring and system alerts.
  • Governance Policies: Manages power and clock settings for GPUs.
  • Diagnostics: Provides a suite of tools for verifying GPU health.
  • Broad Platform Support: Supports Linux on x86_64, Arm, and POWER (ppc64le) architectures.
  • API Support: Provides libraries and source examples in C, Python, and Go.

Related

  • Project
    NVIDIA/dcgm-exporter

    A GPU metrics exporter for Prometheus that leverages NVIDIA DCGM to provide real-time telemetry on GPU health and performance.

  • Project
    leptonai/gpud

    GPUd is a GPU monitoring and management tool that ensures efficiency and reliability for AI/ML workloads by tracking critical hardware metrics and detecting errors.

  • Project
    psalias2006/gpu-hot

    A lightweight, self-hosted real-time NVIDIA GPU monitoring dashboard that supports single-machine or multi-node cluster tracking.

  • Project
    NVIDIA/k8s-device-plugin

    The official NVIDIA device plugin for Kubernetes that enables the cluster to automatically detect, monitor, and allocate NVIDIA GPUs to containers.

  • Project
    utkuozdemir/nvidia_gpu_exporter

    A Prometheus exporter for NVIDIA GPUs that collects metrics via nvidia-smi or NVML, enabling hardware monitoring for consumer GPUs, home labs, and restricted environments.