borglab/gtsfm

What it solves

GTSfM provides an end-to-end Structure-from-Motion (SfM) pipeline to reconstruct 3D scenes from a collection of 2D images. It specifically addresses the need for scalable, distributed computation in 3D reconstruction, allowing the process to be parallelized across multiple machines or workers to handle larger datasets more efficiently.

How it works

The project is built on top of GTSAM and uses Dask for native parallel computation. It follows a modular architecture where different components of the SfM pipeline can be swapped:

• Front-End: Handles keypoint detection, feature description (using tools like SIFT or SuperPoint), matching (e.g., SuperGlue), and correspondence verification.
• Averaging: Implements rotation and translation averaging to establish initial poses.
• Data Association: Performs 3D point triangulation from 2D tracks.
• Bundle Adjustment: Refines the 3D structure and camera poses.
• Densification: Supports dense Multi-View Stereo (MVS) and Gaussian Splatting for high-fidelity scene representation.

Who it’s for

It is designed for researchers and developers working in computer vision, robotics, and 3D reconstruction who need a high-performance, distributed pipeline capable of processing various dataset formats (such as COLMAP, Argoverse, and Tanks and Temples).

Highlights

• Distributed Processing: Native support for parallel computation via Dask, including multi-machine cluster deployment.
• Modular Design: Abstract base classes allow easy swapping of detectors, descriptors, and matchers.
• Deep Learning Integration: Supports pre-trained weights for SuperPoint, SuperGlue, and PatchmatchNet.
• Broad Compatibility: Includes loaders for numerous industry-standard datasets and provides a conversion script for Nerfstudio.
• Visualization: Integrated support for Open3D and COLMAP GUI for viewing reconstructed scenes.