koide3/direct_visual_lidar_calibration

What it solves

This project provides a toolbox for calibrating the extrinsic relationship between a LiDAR sensor and a camera. It eliminates the need for specialized calibration targets (like checkerboards) and manual initial guesses, making the process automatic and more efficient.

How it works

The toolbox uses a pixel-level direct registration algorithm. Instead of relying on edges or specific targets, it leverages the existing structure and texture of the environment to align the LiDAR point cloud with the camera image. It is designed to be generalizable, supporting various LiDAR types (spinning or non-repetitive scans) and camera projection models (pinhole, fisheye, and omnidirectional).

Who it’s for

Engineers and researchers working with robotics, autonomous vehicles, and 3D perception systems that require precise alignment between depth sensors and visual sensors.

Highlights

• Target-less: Calibrates using environment texture and structure rather than specific targets.
• Single-shot: Requires only one pair of LiDAR point cloud and camera image for a basic calibration.
• Automatic: No initial guess is required to start the process.
• Generalizable: Compatible with multiple camera models and LiDAR scan types.
• Robust: Uses direct pixel-level registration for higher accuracy than edge-based methods.