fastvideo/gpu-camera-sample

Image processing software on GPU (Windows, Linux, ARM) for real time machine vision camera applications. Performance benchmarks and Glass-to-Glass time measurements. MIPI CSI cameras support. RAW2RGB processing on CUDA with 16-bit ISP. Software for Jetson.

What it solves

It provides a high-performance image processing pipeline for machine vision cameras, enabling real-time raw image capture and processing on NVIDIA GPUs. It solves the bottleneck of CPU-based image processing by offloading computationally intensive tasks like debayering, denoising, and encoding to the GPU, achieving processing speeds up to 4 GPix/s.

How it works

The application uses a multithreaded architecture to handle raw image acquisition from various camera SDKs (such as XIMEA, FLIR, and Basler) or PGM files. Data is moved from a memory ring buffer to an NVIDIA GPU, where it undergoes a comprehensive ISP (Image Signal Processor) pipeline including raw data conversion, bad pixel correction, white balance, debayering (using algorithms like HQLI or MG), and color correction. The processed frames are then rendered via OpenGL or encoded into JPEG, MJPEG, H.264, or HEVC for storage or streaming.

Who it’s for

It is designed for developers and engineers working with machine vision, industrial cameras, and NVIDIA Jetson platforms who need a tunable, high-precision (16/32-bit) image processing pipeline without relying on fixed-function hardware ISPs.

Highlights

  • Extreme Throughput: Capable of reaching up to 4 GPix/s for color cameras on high-end GPUs like the RTX 4090.
  • Broad Camera Support: Compatible with XIMEA, FLIR, Basler, Lucid Vision Labs, and others via direct SDKs or GenICam.
  • Comprehensive ISP Pipeline: Includes advanced features like NLM denoisers, multiple debayering algorithms, and linear/sRGB gamma correction.
  • Low Latency: Includes built-in tools for measuring glass-to-glass latency, with benchmarks showing ~35–40 ms for specific setups.
  • Cross-Platform: Supports Windows, Linux, and NVIDIA Jetson (ARM64).

Related

  • Project
    NVIDIA-NeMo/Curator

    A GPU-accelerated data curation framework for building repeatable pipelines to load, filter, deduplicate, and transform massive multimodal datasets for AI training.

  • Project
    CVCUDA/CV-CUDA

    CV‑CUDA is NVIDIA’s open‑source GPU‑accelerated computer‑vision library. It provides fast, scalable image/video operators (resize, filter, color conversion, etc.) that run entirely on the GPU and integrate with C/C++ and Python AI frameworks. Install via pre‑built wheels (`cvcuda-cu12` or `cvcuda-cu13`) for Linux, decode images directly to GPU with `nvimgcodec`, and chain CV‑CUDA operators before feeding data to deep‑learning models. The library targets modern NVIDIA GPUs (Turing and newer) and is used in production systems such as Microsoft Bing visual search and Tencent Cloud video processing.

  • Project
    ros-perception/image_pipeline

    A ROS 2 package that processes raw camera images into a usable format for higher-level vision processing in robotics.

  • Project
    rapidsai/cucim

    cuCIM is a GPU-accelerated computer vision and image processing library designed for large, multidimensional images used in biomedical, geospatial, and remote sensing applications.

  • Project
    NVIDIA/CUDALibrarySamples

    A collection of GPU-accelerated examples demonstrating the use of NVIDIA CUDA libraries for high-performance math, image processing, and linear algebra.