anderskm/gputil

What it solves

GPUtil solves the problem of programmatically identifying and selecting available NVIDIA GPUs on a machine. This is particularly useful for deep learning tasks where a user needs to assign a workload to a GPU that isn't already heavily loaded or out of memory, preventing resource conflicts and crashes.

How it works

The module acts as a Python wrapper around the nvidia-smi command-line tool. It queries the system for all available NVIDIA GPUs and retrieves their current status, including memory consumption and load. It then filters these GPUs based on user-defined thresholds for maximum load and memory usage to determine which devices are "available."

Who it’s for

It is designed for developers and researchers working with deep learning libraries (such as TensorFlow or Caffe) who need to dynamically allocate GPU resources across multiple devices.

Highlights

• Flexible Selection: Supports ordering available GPUs by ID, load, or memory usage, or selecting them randomly.
• Resource Monitoring: Provides a showUtilization() function to quickly print the current status of all GPUs.
• GPU Metadata: Accesses detailed hardware information including UUIDs, serial numbers, driver versions, and display status.
• Integration Examples: Includes concrete patterns for masking GPUs in TensorFlow using CUDA_VISIBLE_DEVICES and selecting devices in Caffe.