zju3dv/DetectorFreeSfM

Code for "Detector-Free Structure from Motion", CVPR 2024

What it solves

It addresses the limitations of traditional Structure from Motion (SfM) pipelines that rely on feature detectors to find matching points between images. By removing the need for a detector, the system can more robustly reconstruct 3D scenes from a set of 2D images, especially in challenging environments.

How it works

The project implements a "detector-free" approach to 3D reconstruction. It utilizes coarse matchers (such as LoFTR, ASpanFormer, or MatchFormer) to establish initial correspondences and then employs a multi-view matching refiner to improve the accuracy of these matches. The final results are output in COLMAP format for visualization and further processing.

Who it’s for

Computer vision researchers and engineers working on 3D reconstruction, photogrammetry, and spatial AI who need a more robust way to handle image matching without relying on traditional feature detection.

Highlights

  • Award-winning performance: Ranked 1st in the Image Matching Challenge 2023.
  • Flexible matching: Supports multiple coarse matchers including LoFTR, ASpanFormer, and MatchFormer.
  • Multi-view refinement: Includes a dedicated trainer for a multi-view matching refiner to increase precision.
  • COLMAP integration: Outputs results in a standard format compatible with COLMAP GUI for easy visualization.

Related

  • Project
    nipreps/fmriprep

    A robust preprocessing pipeline for fMRI data that automates motion correction, normalization, and brain extraction using a combination of state-of-the-art neuroimaging tools.

  • Project
    mkturkcan/DART

    A training-free framework that converts SAM3 into a real-time multi-class open-vocabulary detector using TensorRT optimization and distilled backbones.

  • Project
    cs-chan/Total-Text-Dataset

    A word-level English curved text dataset and evaluation framework for improving the detection of arbitrary-shaped text in natural scene images.

  • Project
    zju3dv/LoG

    LoG is a framework for training and real-time rendering of highly realistic urban-scale 3D models using Gaussian Splatting on a single RTX 4090 GPU.