rapidsai/rmm

NVIDIA RMM (RAPIDS Memory Manager)

What it is – RMM is a C++/Python library that provides a common, flexible interface for allocating GPU memory. It lets developers plug in different allocation strategies (plain cudaMalloc, pooled allocators, managed memory, etc.) and use them from both C++ code and Python via the rmm package.

Why it matters – In high‑performance GPU workloads, the cost of repeatedly calling cudaMalloc/cudaFree can dominate runtime. RMM’s pool and binning allocators dramatically reduce that overhead and support stream‑ordered allocation, meaning memory can be reused on the same CUDA stream without extra synchronization.

Key features

• Multiple memory‑resource implementations – cuda_memory_resource, managed_memory_resource, pool_memory_resource, fixed_size_memory_resource, binning_memory_resource.
• Stream‑ordered allocation – allocate and free on a specific CUDA stream, enabling fast reuse.
• Per‑device default resources – you can set a global allocator for each GPU, or override per‑allocation.
• C++ API – thin wrappers around CUDA’s allocation functions, plus RAII containers like rmm::device_buffer, rmm::device_uvector, and rmm::device_vector.
• Python bindings – installable via pip/conda (librmm-cuXX and rmm-cuXX).
• ABI versioning – multiple versions can coexist in the same process.

Typical use‑case

rmm::mr::cuda_memory_resource cuda_mr;
auto pool = rmm::mr::pool_memory_resource{cuda_mr,
                                          rmm::percent_of_free_device_memory(50)};
auto prev = rmm::mr::set_current_device_resource(pool);
// Subsequent allocations use the pool automatically
rmm::device_buffer buf(1024, cuda::stream_ref{cudaStreamDefault}, pool);

In Python you would import rmm; rmm.reinitialize(pool_allocator=True) to switch to a pooled allocator.

How to get it – Install the pre‑built wheels (pip install librmm-cu13 rmm-cu13 for CUDA 13, etc.) or build from source using CMake and the provided build.sh. The repo includes a full set of docs, tutorials, and CMake integration helpers.

Who uses it – Part of NVIDIA’s RAPIDS ecosystem; downstream projects such as cuDF, cuML, and other GPU‑accelerated data‑science libraries rely on RMM for efficient memory handling.