facebookresearch/map-anything

What it solves

MapAnything provides a universal framework for metric 3D reconstruction. It addresses the challenge of performing various 3D tasks—such as multi-image structure-from-motion (SfM), multi-view stereo, monocular metric depth estimation, and registration—using a single, unified feed-forward model rather than separate specialized tools for each task.

How it works

At its core, the project uses an end-to-end trained transformer model that regresses the factored metric 3D geometry of a scene. It is highly flexible with its inputs, allowing users to provide any combination of images, camera calibration (intrinsics or ray directions), camera poses, or depth maps. The framework is designed modularly, meaning it can wrap other 3D reconstruction models (like DUSt3R, MASt3R, and VGGT) through a unified interface, ensuring all outputs follow a consistent format (e.g., 3D points, camera poses, and confidence scores).

Who it’s for

This framework is intended for researchers and developers working in 3D computer vision, robotics, and spatial AI who need a versatile tool for metric 3D reconstruction across different input modalities.

Highlights

• Universal Model: A single feed-forward model supporting over 12 different 3D reconstruction tasks.
• Multi-Modal Flexibility: Supports any combination of images, calibration, depth, and poses as inputs.
• Modular Design: Unified interface for interchangeable use of various external 3D reconstruction models.
• Memory Efficient: Includes a memory-efficient inference mode capable of handling up to 2,000 views on high-end GPUs.
• Comprehensive Stack: Provides a complete pipeline including data processing, training, inference, and profiling.