NVIDIA-AI-IOT/Lidar_AI_Solution

A project demonstrating Lidar related AI solutions, including three GPU accelerated Lidar/camera DL networks (PointPillars, CenterPoint, BEVFusion) and the related libs (cuPCL, 3D SparseConvolution, YUV2RGB, cuOSD,).

What it solves

This project provides a highly optimized suite of CUDA and TensorRT implementations for 3D Lidar-based perception in self-driving vehicles. It addresses the performance bottlenecks associated with sparse convolutions, point cloud processing, and multi-modal sensor fusion (Lidar and Camera), enabling faster inference and lower memory usage on NVIDIA hardware.

How it works

The solution implements several specialized components:

  • Inference Engines: Optimized CUDA kernels and TensorRT integrations for popular 3D detection architectures like CenterPoint, BEVFusion, and PointPillars.
  • Sparse Convolution: A dedicated, lightweight inference engine for 3D sparse convolutional networks that operates independently of TensorRT using INT8/FP16 precision.
  • Point Cloud Library (cuPCL): GPU-accelerated operations for point cloud registration (ICP, NDT), filtering, segmentation, and clustering.
  • Data Pipeline: High-performance kernels for YUV-to-RGB conversion, ROI processing, and voxelization to prepare raw sensor data for AI models.
  • Visualization: A CUDA-based on-screen display (cuOSD) library for drawing bounding boxes and text directly via GPU kernels.

Who it’s for

This is designed for engineers and researchers working on autonomous driving systems, robotics, and 3D perception who need to deploy high-performance Lidar-based AI models on NVIDIA GPUs.

Highlights

  • Optimized Architectures: High-fidelity implementations of BEVFusion, CenterPoint, and PointPillars.
  • Memory Efficiency: Low memory footprint for sparse convolutions (approx. 422-426MB).
  • Hardware Acceleration: Extensive use of CUDA kernels for pre-processing, post-processing, and visualization to minimize CPU-GPU data transfer.
  • Quantization Support: Includes Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) solutions.
  • Integrated Tooling: Includes a GPU-accelerated point cloud library (cuPCL) and a fast image conversion utility.

Related

  • Project
    NVIDIA/TensorRT-LLM

    An open-source library from NVIDIA that optimizes inference for LLMs and visual generation models using specialized kernels and an efficient runtime to maximize throughput and latency on NVIDIA GPUs.

  • Project
    NVIDIA/TensorRT

    NVIDIA TensorRT is a high-performance deep learning inference library and SDK that accelerates AI model deployment on NVIDIA GPUs.

  • Project
    cyrusbehr/YOLOv8-TensorRT-CPP

    A C++ implementation of YOLOv8 using TensorRT for high-performance GPU inference, supporting object detection, semantic segmentation, and pose estimation.

  • Project
    NVlabs/curobo

    A CUDA-accelerated library for robot motion generation that provides GPU-parallel algorithms for kinematics, collision checking, and trajectory optimization for high-DoF robots.

  • Project
    Tencent/hpc-ops

    A production-grade operator library for LLM inference that provides highly optimized CUDA kernels for attention, MoE, and sampling to reduce serving latency.