DALI: a GPU-accelerated data loading and pre-processing library that eliminates CPU bottlenecks in deep learning pipelines

What it solves

NVIDIA DALI (Data Loading Library) eliminates the CPU bottleneck in deep learning pipelines. In traditional workflows, data loading and preprocessing (like decoding, cropping, and resizing) are handled by the CPU, which often limits the overall performance and scalability of training and inference.

How it works

DALI offloads data preprocessing tasks to the GPU. It uses a dedicated execution engine designed to maximize throughput via transparent prefetching, parallel execution, and batch processing. It provides a functional Python API and supports both a "Pipeline mode" for defined graphs and a "Dynamic mode" for more flexible execution.

Who it’s for

Deep learning practitioners and researchers who work with image, video, and audio data and need to maximize GPU utilization by accelerating their input pipelines across frameworks like PyTorch, TensorFlow, JAX, and PaddlePaddle.

Highlights

• Multi-format support: Supports a wide range of formats including JPEG, JPEG 2000, WAV, FLAC, OGG, H.264, VP9, HEVC, LMDB, and TFRecord.
• Framework portable: Works as a drop-in replacement for data loaders in PyTorch, TensorFlow, JAX, and PaddlePaddle.
• Hardware acceleration: Supports both CPU and GPU execution and scales across multiple GPUs.
• Direct data path: Enables a direct path between storage and GPU memory via GPUDirect Storage.
• Extensible: Allows developers to create custom pipelines and operators.
• Triton integration: Integrates easily with NVIDIA Triton Inference Server.