NVIDIA/DALI
A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
What it solves
NVIDIA DALI (Data Loading Library) eliminates the CPU bottleneck in deep learning pipelines. In traditional workflows, data loading and preprocessing (like decoding, cropping, and resizing) are handled by the CPU, which often limits the overall performance and scalability of training and inference.
How it works
DALI offloads data preprocessing tasks to the GPU. It uses a dedicated execution engine designed to maximize throughput via transparent prefetching, parallel execution, and batch processing. It provides a functional Python API and supports both a "Pipeline mode" for defined graphs and a "Dynamic mode" for more flexible execution.
Who it’s for
Deep learning practitioners and researchers who work with image, video, and audio data and need to maximize GPU utilization by accelerating their input pipelines across frameworks like PyTorch, TensorFlow, JAX, and PaddlePaddle.
Highlights
- Multi-format support: Supports a wide range of formats including JPEG, JPEG 2000, WAV, FLAC, OGG, H.264, VP9, HEVC, LMDB, and TFRecord.
- Framework portable: Works as a drop-in replacement for data loaders in PyTorch, TensorFlow, JAX, and PaddlePaddle.
- Hardware acceleration: Supports both CPU and GPU execution and scales across multiple GPUs.
- Direct data path: Enables a direct path between storage and GPU memory via GPUDirect Storage.
- Extensible: Allows developers to create custom pipelines and operators.
- Triton integration: Integrates easily with NVIDIA Triton Inference Server.