facebookresearch/vggt-omega

[CVPR 2026 Oral] VGGT Omega

What it solves

VGGT-$\Omega$ addresses the challenge of estimating camera poses and depth from a set of images. It allows users to reconstruct a 3D scene (as a point cloud and camera positions) from 2D images without requiring manual intervention or complex traditional pipelines.

How it works

The project provides a pretrained transformer-based model that takes multiple images as input and predicts camera extrinsics, intrinsics, and depth maps. It uses a specific encoding for camera poses and provides a mechanism to convert these encodings back into usable camera parameters. A specialized version of the model also supports text alignment, allowing for embeddings that can be aligned with text descriptions.

Who it’s for

This tool is designed for researchers and developers working in computer vision, 3D reconstruction, and spatial AI who need to efficiently estimate camera parameters and depth from image sequences or single images.

Highlights

  • End-to-end inference: Predicts camera poses and depth maps directly from images.
  • 3D Visualization: Includes a Gradio demo that can visualize depth-unprojected point clouds and predicted cameras as a GLB scene.
  • Flexible Resolution: Supports different input resolutions (e.g., 256 or 512) depending on the model checkpoint used.
  • Text Alignment: Offers a specific checkpoint for aligning visual spatial data with text embeddings.

Related

  • Project
    MIT-SPARK/VGGT-SLAM

    A real-time dense feed-forward scene reconstruction system for 3D mapping and localization, featuring optional open-set 3D object detection.

  • Project
    microsoft/MoGe

    MoGe is a model for recovering accurate 3D geometry, including metric depth and point maps, from single open-domain images.

  • Project
    facebookresearch/mvdust3r

    A single-stage 3D scene reconstruction tool that creates 3D point clouds and camera poses from sparse RGB views in about 2 seconds without requiring pre-known camera poses.

  • Project
    ByteDance-Seed/Depth-Anything-3

    A model that predicts spatially consistent 3D geometry, depth maps, and camera poses from single or multiple images using a unified depth-ray representation.

  • Project
    facebookresearch/VLM3

    VLM³ is a framework that enables standard Vision Language Models to perform 3D vision tasks like depth estimation and camera pose estimation using only text-based outputs and data scaling.