cre185/InstantSfM

Official code for paper "InstantSfM: Towards GPU-Native SfM for the Deep Learning Era"

What it solves

InstantSfM addresses the efficiency bottlenecks of traditional Structure from Motion (SfM) pipelines, which are often slow and not fully optimized for modern GPU hardware. It provides a GPU-native approach to reconstruct 3D structures from a collection of 2D images, making the process faster and more compatible with deep learning workflows.

How it works

The project implements a full SfM pipeline consisting of three primary stages:

  1. Feature Extraction and Matching: Identifies key points across images (using COLMAP by default).
  2. Global SfM and Bundle Adjustment: Computes the 3D structure and camera poses using GPU-native operations to optimize the reconstruction.
  3. 3D Gaussian Splatting (3DGS): Provides integrated support to train 3DGS models based on the reconstructed data.

It utilizes a SIMD-friendly data structure to improve processing speed and integrates with tools like bae for bundle adjustment in eager mode.

Who it’s for

This tool is designed for researchers and developers working in computer vision, robotics, and 3D reconstruction who need a high-performance, GPU-accelerated pipeline to turn images into 3D models.

Highlights

  • GPU-Native Architecture: Optimized for NVIDIA GPUs to accelerate the SfM process.
  • End-to-End Pipeline: Covers everything from feature extraction to 3DGS training.
  • Interactive Visualization: Includes a GUI to visualize the reconstruction process in real-time or playback recorded reconstructions.
  • Metric Depth Support: Includes tools for metric scale depth estimation from videos using Video Depth Anything.

Related

  • Project
    borglab/gtsfm

    An end-to-end, distributed Structure-from-Motion pipeline based on GTSAM and Dask that reconstructs 3D scenes from 2D images with native parallel computation support.

  • Project
    colmap/colmap

    A general-purpose Structure-from-Motion and Multi-View Stereo pipeline for reconstructing 3D structures from image collections.

  • Project
    mapillary/OpenSfM

    A Structure from Motion library in Python that reconstructs 3D scenes and camera poses from multiple images, integrating sensor data for geographical alignment.

  • Project
    Linketic/CityGaussian

    A framework for high-quality, large-scale 3D scene reconstruction using Gaussian Splatting, featuring multi-GPU support and geometrically accurate rendering.

  • Project
    cvg/vidmap

    An offline Structure-from-Motion system for video that estimates camera poses and 3D maps by exploiting temporal tracks, loop closures, and metric depth.