NVIDIA/cudf

NVIDIA cuDF – GPU‑accelerated DataFrames

What it is – cuDF is a set of libraries that let you work with tabular data (CSV, Parquet, etc.) on an NVIDIA GPU. It mirrors the pandas API, so you can drop‑in replace pandas code and get the same operations (read, filter, group‑by, joins, etc.) running on the GPU instead of the CPU. The core is a C++ library (libcudf) with Arrow‑compatible columnar structures, and Python bindings are provided through pylibcudf and the high‑level cudf package. Additional adapters let cuDF power Polars (cudf‑polars) and Dask (dask‑cudf).

Why it matters for AI/ML – Most data‑science pipelines start with large tables that need cleaning, reshaping, and aggregating before they can be fed to models. cuDF accelerates those steps, often cutting preprocessing time from minutes to seconds, which speeds up model iteration and enables larger datasets.

Key components

• libcudf – CUDA C++ core with Arrow‑compatible columnar containers and low‑level algorithms.
• pylibcudf – Cython bindings exposing the core to Python.
• cudf – Pandas‑compatible DataFrame library; also provides cudf.pandas for zero‑code‑change acceleration of existing pandas scripts.
• cudf‑polars – GPU engine for the Polars DataFrame library.
• dask‑cudf – GPU‑backed Dask DataFrames for distributed workloads.

Installation (choose one)

# via pip – pick the suffix that matches your CUDA version
pip install cudf-cu13   # for CUDA 13
pip install dask-cudf-cu13
# …or for CUDA 12
pip install cudf-cu12

# via conda – use the RAPIDS channel
conda install -c rapidsai cudf dask-cudf

For development builds add the nightly channel (-c rapidsai-nightly).

Quick example – Same code you would write for pandas, but it runs on the GPU:

import cudf

# Load a Parquet file directly into GPU memory
df = cudf.read_parquet('data.parquet')

# Drop rows with any nulls, group by two columns, and compute the mean
result = df.dropna().groupby(['A', 'B']).mean()
print(result)

You can also run existing pandas scripts unchanged:

python -m cudf.pandas my_script.py

Or in a Jupyter notebook:

%load_ext cudf.pandas
import pandas as pd   # now runs on GPU

For Polars users:

import polars as pl
lf = pl.scan_parquet('data.parquet')
out = lf.drop_nulls().group_by(['A', 'B']).mean().collect(engine='gpu')

Community & contribution – Bugs and feature requests go through the GitHub issue tracker. Discussions happen on the RAPIDS Slack workspace. Contributions are welcomed via the standard NVIDIA RAPIDS contribution guide.

---

cuDF is a genuine, production‑grade GPU data‑processing library that fits squarely into modern AI/ML pipelines.