LiteReality/LiteReality

[NeurIPS 2025] LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans

What it solves

LiteReality transforms raw RGB-D scans (captured via devices like LiDAR-equipped iPhones) into high-quality, graphics-ready 3D scenes. It bridges the gap between messy real-world scans and clean, renderable 3D models by reconstructing scenes with physically-based rendering (PBR) materials.

How it works

The system processes scans through a multi-stage pipeline:

  1. Object Reconstruction: It identifies and reconstructs individual objects within the scanned scene.
  2. Material Painting: It assigns realistic PBR materials to these objects, utilizing a material database and multimodal models (including CLIP, DinoV2, Qwen-VL, and SAM) to ensure visual accuracy.
  3. Scene Integration: The final output is integrated into a complete 3D scene, exported as GLB or native Blender files for professional rendering.

Who it’s for

This tool is designed for researchers and developers in computer vision, graphics, and robotics who need to convert real-world indoor environments into editable, high-fidelity 3D digital twins.

Highlights

  • Graphics-Ready Output: Produces GLTF/GLB and Blender files with full PBR materials.
  • Multimodal Integration: Leverages advanced vision-language models for material assignment.
  • Mobile Compatibility: Works with scans captured via Apple RoomPlan and the 3D Scanner App.
  • End-to-End Pipeline: Handles everything from raw RGB-D input to final rendered visualization.

Related

  • Project
    LiteReality/LiteReality-Agent

    LiteReality‑Agent is an open‑source toolkit that converts a phone‑captured RGB‑D scan (via the LiteReality iOS app) into a fully‑articulated, graphics‑ready indoor 3D scene. An LLM‑driven agent coordinates vision models (TRELLIS, GroundingDINO) and Blender to generate, place, and polish objects, producing GLB/BLEND assets usable in game engines, AR/VR, or robotics. It runs on macOS or Linux, can use Modal’s cloud GPUs or a local ≥24 GB GPU, and requires API keys for image generation and the LLM.

  • Project
    realsee-developer/RealSee3D

    A large-scale multi-view RGB-D dataset of 10,000 indoor scenes designed for training AI models in 3D reconstruction and semantic segmentation.

  • Project
    theICTlab/3DUNDERWORLD-SLS-GPU_CPU

    A structured light scanning tool that reconstructs 3D point clouds from images of objects lit by patterned light, featuring both CPU and GPU acceleration.

  • Project
    fastvideo/gpu-camera-sample

    A high-performance sample application for real-time GPU-accelerated raw image processing and ISP pipelines for industrial cameras on NVIDIA hardware.

  • Project
    CERN/TIGRE

    A GPU-accelerated toolbox for fast and accurate 3D tomographic reconstruction using Python and MATLAB, supporting a wide range of iterative algorithms and flexible CT geometries.