psalias2006/gpu-hot

What it solves

GPU Hot provides a lightweight, self-hosted dashboard for real-time monitoring of NVIDIA GPUs. It eliminates the need for complex monitoring setups by providing a web-based interface to track hardware metrics and process usage across single machines or entire clusters of GPU servers.

How it works

The tool uses a FastAPI backend to collect metrics via NVML (NVIDIA Management Library) or nvidia-smi as a fallback for older hardware. It transmits this data in real-time via WebSockets to a web frontend that renders historical charts and status cards. For multi-node setups, it can operate in "hub mode," where a central hub aggregator collects and displays data from multiple remote GPU nodes.

Who it’s for

It is designed for developers, ML engineers, and system administrators who need to monitor GPU utilization, temperature, and power draw during model training or inference, especially when managing multiple GPU servers.

Highlights

• Real-time Monitoring: Sub-second metric updates for utilization, temperature, memory, and power draw.
• Cluster Scaling: Ability to scale from a single GPU to over 100 GPUs across multiple nodes using a hub-and-spoke architecture.
• Process Tracking: Monitors active GPU processes, including PIDs and memory usage.
• System Integration: Tracks host-level metrics like CPU, RAM, swap, and disk usage alongside GPU data.
• Efficient Resource Use: Automatically pauses polling when no clients are connected to minimize CPU overhead.