NVIDIA/cudf-spark

NVIDIA cuDF for Apache Spark plugin - accelerate Apache Spark with GPUs

What it solves

It accelerates Apache Spark processing by leveraging GPUs, reducing the time required for data processing tasks.

The project provides a plugin library that uses the cuDF (CUDA DataFrame) libraries to offload Spark processing to NVIDIA GPUs.

Data engineers and developers using Apache Spark who want to increase their data processing performance via GPU acceleration.

GPU-accelerated processing for Apache Spark via cuDF libraries.
Aims for bit-for-bit identical results compared to standard Apache Spark.
Provides APIs for zero-copy data transfer into other GPU-enabled applications, such as ML libraries like XGBoost.
Supports GPU-accelerated User Defined Functions (UDFs).

Project
NVIDIA/cudf
A GPU-accelerated DataFrame library for tabular data processing that mirrors the pandas API and integrates with Polars and Dask.
Project
rapidsai/cugraph
A collection of GPU-accelerated graph analytics packages that enable the creation and manipulation of graphs and the execution of scalable graph algorithms.
Project
NVIDIA/CUDALibrarySamples
A collection of GPU-accelerated examples demonstrating the use of NVIDIA CUDA libraries for high-performance math, image processing, and linear algebra.
Project
pyscf/gpu4pyscf
A GPU acceleration plugin for PySCF that speeds up quantum chemistry simulations, including Hartree-Fock and Density Functional Theory calculations.
Project
neka-nat/cupoch
A GPU-accelerated library for rapid 3D data processing in robotics, providing high-performance point cloud algorithms, collision avoidance, and path planning.