realsee-developer/RealSee3D

RealSee3D: A multi-view RGB-D dataset combining real-world captures and procedurally generated scenes, with extensible annotations for diverse 3D vision research.

What it solves

It provides a massive, high-quality dataset of indoor environments to help researchers train and test AI models for 3D reconstruction, depth estimation, and semantic understanding of indoor spaces.

How it works

Realsee3D combines 1,000 real-world residential scenes captured via 3D LiDAR cameras with 9,000 procedurally generated scenes based on designer-curated templates. The dataset includes multi-view RGB-D panoramic images, camera extrinsics, and pixel-wise semantic segmentation maps, offering complete room-level coverage across nearly 100,000 rooms.

Who it’s for

Computer vision researchers and AI engineers working on indoor 3D scene reconstruction, panoramic image analysis, and spatial AI.

Highlights

  • Massive Scale: Contains 10,000 unique indoor scenes and nearly 300,000 RGB-D pairs.
  • Hybrid Data: Mixes real-world captures with synthetic data to ensure a wide variety of furniture styles and layouts.
  • Rich Annotations: Includes semantic segmentation and covisibility score matrices.
  • Proven Utility: Used to train Argus, a state-of-the-art network for metric panoramic 3D reconstruction.

Related

  • Project
    LiteReality/LiteReality

    LiteReality converts RGB-D scans from mobile devices into graphics-ready 3D scenes with physically-based rendering materials for high-fidelity reconstruction.

  • Project
    LiteReality/LiteReality-Agent

    An agentic toolkit that transforms indoor 3D scans into interactable, graphics-ready 3D scenes with articulated assets and refined materials.

  • Project
    nv-tlabs/NKSR

    Neural Kernel Surface Reconstruction (NKSR) is a method for creating high-quality 3D implicit surfaces from large-scale, sparse, and noisy point clouds, capable of handling millions of points in seconds.

  • Project
    VisionDepth/VisionDepth3D

    A comprehensive 3D creation suite that uses AI depth estimation and stereo warping to convert 2D images and videos into VR-ready 3D stereo content.

  • Project
    microsoft/LatentSpatialMemory

    A framework for video world models that stores 3D scene content as latent tokens to improve generation speed and reduce memory overhead by avoiding repeated RGB rendering.