Yarroudh/segment-lidar

What it solves

Aerial LiDAR point clouds are dense and unstructured, making it difficult to automatically separate out individual objects like buildings, trees, or vehicles. Manual annotation is slow and error-prone. This package automates the task by converting 3D point clouds into 2D images, running Meta's powerful SAM 3 segmentation model, and mapping the detected object masks back onto the original points, giving each point an instance ID.

How it works

The package projects a point cloud into a 2D image using either a top-down view or a user-defined pinhole camera. It then runs SAM 3 on that image to find objects. The user can either let the model automatically detect all masks, or guide it with text prompts (e.g., "building") and/or box prompts (positive and negative examples). After segmentation, the labels from the image are transferred back to the point cloud, assigning each point an instance ID. For large areas, ground filtering can be applied first to remove ground points, and it is also possible to reuse previously saved label rasters to avoid rerunning the model.

Who it’s for

Researchers, surveyors, and developers working with aerial LiDAR data who need to extract object instances without manually labeling each point. It is especially useful for urban mapping, forestry, and 3D city modeling. Users should have Python 3.10 or newer, a CUDA GPU (recommended for SAM 3 inference), and access to Meta's gated SAM 3 model via Hugging Face (requires a token).

Highlights

• Uses Meta's SAM 3, a state-of-the-art segmentation model, adapted for 3D point clouds.
• Supports automatic mask generation, text prompts, and box prompts.
• Projects point clouds into top-view or custom pinhole camera images.
• Can apply saved label rasters to new point clouds without rerunning inference.
• Optional ground filtering (CSF) and output to LAS/LAZ with instance IDs.
• Includes Docker support for easy deployment with GPU.