NVIDIA/gdrcopy

What it solves

GDRCopy is a low-latency GPU memory copy library designed to reduce the overhead associated with moving data between the CPU and GPU. It addresses the high latency typically found in standard copy operations (like cudaMemcpy), which can incur overheads of 6-7 microseconds, by allowing the CPU to drive the copy process directly.

How it works

The library leverages NVIDIA GPUDirect RDMA technology to create user-space mappings of GPU memory. This allows the CPU to manipulate GPU memory as if it were plain host memory.

It supports two backends:

1. gdrdrv: A dedicated kernel module for mapping GPU memory.
2. DMA-BUF mmap: A backend for environments where the kernel module is not installed, utilizing the CUDA driver to export GPU memory as a Linux dma-buf for mapping via mmap().

Performance is characterized by fast Host-to-Device (H-D) transfers due to write-combining, but slower Device-to-Host (D-H) transfers because the GPU BAR cannot be prefetched.

Who it’s for

Developers working with NVIDIA Data Center or RTX GPUs who require extremely low-latency data transfers between the CPU and GPU for performance-critical applications.

Highlights

• Low Overhead: CPU-driven copies significantly reduce latency compared to standard CUDA memory copies.
• Flexible Mapping: Provides infrastructure to create user-space mappings of GPU memory.
• Multiple Backends: Supports both a custom kernel module and a DMA-BUF mmap fallback.
• Benchmarking Tools: Includes a suite of tests for sanity checks, bandwidth calculation, and round-trip ping-pong latency.