NVIDIA/nvbench

CUDA Kernel Benchmarking Library

What it solves

NVBench simplifies the process of benchmarking CUDA kernels. It provides a structured way to measure execution time, memory bandwidth, and throughput, removing the manual effort required to set up complex performance tests for GPU kernels.

How it works

It is a C++17 library that allows developers to define benchmarks for specific kernels. It uses an "axis" system to perform parameter sweeps, exploring different configurations (such as data sizes or types) to find optimal settings. The library supports different measurement modes, including "Cold Measurements" (clearing L2 cache) and "Batch Measurements" (running multiple iterations) to ensure accurate timing.

Who it’s for

CUDA developers and performance engineers who need to tune individual GPU kernels and perform regression testing on their hardware.

Highlights

  • Parameter Sweeps: Flexible system for exploring kernel configuration spaces using dynamic numbers, strings, or static types.
  • Throughput Analysis: Automatically calculates item throughput (elements/second) and global memory bandwidth usage relative to peak bandwidth.
  • Runtime Customization: A rich CLI allows users to redefine parameter axes and select CUDA devices without recompiling.
  • Multiple Measurement Types: Supports cold starts, batch execution, and CPU-only measurements.
  • Flexible Output: Generates results in Markdown or CSV formats.

Related

  • Project
    NVIDIA/nvbandwidth

    A performance benchmarking tool for NVIDIA GPUs that measures memory bandwidth and latency across CPU-to-GPU and GPU-to-GPU links, including multi-node configurations.

  • Project
    NVIDIA/nccl-tests

    A performance and correctness testing suite for the NVIDIA Collective Communications Library (NCCL) used to benchmark multi-GPU and multi-node communication.

  • Project
    eyalroz/cuda-api-wrappers

    A header-only C++ library providing thin, modern wrappers for NVIDIA's CUDA Runtime, Driver, NVRTC, and NVTX APIs to simplify GPU programming and resource management.

  • Project
    kunzmi/managedCuda

    A .NET wrapper for NVIDIA's CUDA Driver API and libraries, enabling C# and Visual Basic developers to integrate GPU acceleration into their applications.

  • Project
    NVIDIA/MatX

    MatX is a header‑only C++20 library that provides NumPy‑style tensor operations with lazy expression graphs. By swapping executors you can run the same code on NVIDIA GPUs, on multithreaded CPUs, or as a single JIT‑fused kernel, achieving near‑hand‑written CUDA performance with far less code.