facebookresearch/mvdust3r

Open source impl of **MV-DUSt3R+ Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds** from Meta Reality Labs. Project page https://mv-dust3rp.github.io/

What it solves

MV-DUSt3R+ enables fast, single-stage 3D scene reconstruction from a small number of RGB images (sparse views). It removes the need for pre-calculated camera poses, allowing the system to reconstruct a scene from images or video without knowing exactly where the cameras were located.

How it works

The project is a finetuned version of the DUSt3R model. It processes multiple RGB images or video frames to predict 3D point clouds and camera poses in a single step. It can handle a variable number of views (ranging from 4 to 12) and provides a confidence threshold to filter out low-quality points in the reconstruction.

Who it’s for

Researchers and developers working in computer vision, 3D reconstruction, and spatial AI, specifically those looking for a high-speed reconstruction pipeline that doesn't require explicit camera calibration or pose estimation as a separate step.

Highlights

  • Pose-free reconstruction: Reconstructs 3D scenes from RGB-only images without requiring camera poses.
  • Single-stage process: Performs reconstruction in a single step rather than a multi-stage pipeline.
  • Extreme speed: Capable of reconstructing scenes in approximately 2 seconds.
  • Versatile input: Supports both multiple static images and video files.
  • Additional capabilities: Supports relative pose estimation and new view synthesis.

Related

  • Project
    hujiecpp/PE3R

    PE3R is a perception-efficient 3D reconstruction system that converts 2D images into semantically understood 3D scenes using zero-shot generalization.

  • Project
    Eyeline-Labs/Vista4D

    Vista4D is a video reshooting framework that uses 4D point clouds to synthesize dynamic scenes from source videos using novel camera trajectories and viewpoints.

  • Project
    WHU-USI3DV/VistaDream

    VistaDream is a training-free framework that reconstructs high-quality 3D scenes from a single-view image by ensuring consistency across generated novel views.

  • Project
    nianticlabs/map-free-reloc

    A reference implementation for map-free visual relocalization that enables metric pose estimation relative to a single image, removing the need for large 3D maps.

  • Project
    microsoft/MoGe

    MoGe is a model for recovering accurate 3D geometry, including metric depth and point maps, from single open-domain images.