realsee-developer/RealSee3D

RealSee3D: A multi-view RGB-D dataset combining real-world captures and procedurally generated scenes, with extensible annotations for diverse 3D vision research.

What it solves

It provides a massive, high-quality dataset of indoor environments to help researchers train and test AI models for 3D reconstruction, depth estimation, and semantic understanding of indoor spaces.

How it works

Realsee3D combines 1,000 real-world residential scenes captured via 3D LiDAR cameras with 9,000 procedurally generated scenes based on designer-curated templates. The dataset includes multi-view RGB-D panoramic images, camera extrinsics, and pixel-wise semantic segmentation maps, offering complete room-level coverage across nearly 100,000 rooms.

Who it’s for

Computer vision researchers and AI engineers working on indoor 3D scene reconstruction, panoramic image analysis, and spatial AI.

Highlights

  • Massive Scale: Contains 10,000 unique indoor scenes and nearly 300,000 RGB-D pairs.
  • Hybrid Data: Mixes real-world captures with synthetic data to ensure a wide variety of furniture styles and layouts.
  • Rich Annotations: Includes semantic segmentation and covisibility score matrices.
  • Proven Utility: Used to train Argus, a state-of-the-art network for metric panoramic 3D reconstruction.

Related

  • Project
    LiteReality/LiteReality

    LiteReality converts RGB-D scans from mobile devices into graphics-ready 3D scenes with physically-based rendering materials for high-fidelity reconstruction.

  • Project
    LiteReality/LiteReality-Agent

    LiteReality‑Agent is an open‑source pipeline that converts phone‑captured RGB‑D scans (via the LiteReality iOS app) into fully articulated, physics‑ready 3‑D indoor scenes. It cleans layout, infers object parts, assigns masses/inertias, generates textures with LLM‑driven image APIs, and exports to MuJoCo URDF/MJCF, Blender, or GLB. The heavy AI models run on Modal’s free GPU service, so even Apple‑silicon Macs can use it. One CLI command (`uv run litereality run <scan>`) performs the whole reconstruction, after a brief `uv`‑based environment setup.

  • Project
    nv-tlabs/NKSR

    Neural Kernel Surface Reconstruction (NKSR) is a method for creating high-quality 3D implicit surfaces from large-scale, sparse, and noisy point clouds, capable of handling millions of points in seconds.

  • Project
    VisionDepth/VisionDepth3D

    VisionDepth 3D is a Windows desktop app that converts 2‑D images or videos into stereoscopic 3‑D using AI depth models, depth‑map blending, RIFE frame‑interpolation, and AI super‑resolution. It offers a free tier (limited length, 1080p, watermarked) and a one‑time $59.99 Pro tier (no limits, 4K+, batch processing, advanced controls). Installation is via an Itch.io “Setup Hub” that provides CUDA (NVIDIA) or DirectML (AMD/Intel) builds. The software is proprietary, but the AI models it downloads are open‑source under their own licenses.

  • Project
    microsoft/LatentSpatialMemory

    A framework for video world models that stores 3D scene content as latent tokens to improve generation speed and reduce memory overhead by avoiding repeated RGB rendering.