ika-rwth-aachen/ros2-depth-anything-v3-trt

What it solves

This project provides a ROS 2 node that enables real-time monocular metric depth estimation and 3D point cloud generation from a single camera feed. It solves the problem of obtaining accurate distance measurements and spatial mapping from standard images using the Depth Anything V3 model, accelerated by NVIDIA TensorRT for high-performance inference.

How it works

The node subscribes to camera images and calibration data (camera info). It processes these images through a TensorRT-accelerated Depth Anything V3 model to produce a dense metric depth map. The system then applies a post-processing pipeline that scales depth based on camera focal length, handles sky pixels to prevent unrealistic distance values, and upscales the depth map to the original image resolution. Finally, it converts the depth map into a PointCloud2 message using the camera's intrinsic parameters.

Who it’s for

This tool is designed for robotics and automated driving developers who use ROS 2 (specifically Jazzy) and NVIDIA GPUs, needing real-time spatial awareness and depth perception for their vehicles or robots.

Highlights

• Real-time Performance: Achieves up to 50 FPS on a Quadro RTX 6000 using the DA3METRIC-LARGE model.
• TensorRT Acceleration: Supports FP16 and FP32 precision for optimized GPU inference.
• Integrated Point Cloud Generation: Directly converts depth maps into 3D point clouds with optional RGB colorization.
• Sky Handling: Includes a specific classification step to identify and cap the depth of sky pixels.
• Docker Support: Provides precompiled Docker images to simplify dependency management for CUDA and TensorRT.