nv-tlabs/lyra

Project Lyra: Open Generative 3D World Models

What it solves

Project Lyra provides a way to generate 3D and 4D scenes from limited visual input. It addresses the challenge of creating consistent, explorable 3D worlds from a single image or video, enabling the transition from 2D generative AI to spatially consistent 3D environments.

How it works

The project consists of two main versions:

  • Lyra 1.0: Uses video diffusion model self-distillation to enable feed-forward 3D and 4D scene generation from a single image or video.
  • Lyra 2.0: Focuses on long-horizon, 3D-consistent generation to create explorable generative 3D worlds.

Who it’s for

This technology is designed for researchers and developers working in 3D scene reconstruction, generative AI, and the creation of virtual environments.

Highlights

  • 3D/4D Generation: Capable of generating 3D scenes and 4D (time-varying) scenes from a single image or video.
  • 3D Consistency: Lyra 2.0 specifically targets long-horizon consistency to ensure the world remains stable as it is explored.
  • Open Implementation: Provides official implementations, training code, and a GUI for Lyra 2.0.

Related

  • Project
    Tencent-Hunyuan/HY-World-2.0

    A multi-modal world model framework that generates and reconstructs editable 3D assets (meshes and Gaussian Splattings) from text, images, or video for use in game engines.

  • Project
    AMAP-ML/DreamX-World

    DreamX-World is a general-purpose interactive world model that generates high-fidelity, controllable simulations allowing users to explore and transform environments via event prompts.

  • Project
    luciddreamer-cvlab/LucidDreamer

    LucidDreamer is a system for domain-free generation of 3D Gaussian Splatting scenes from a single image and a text prompt.

  • Project
    ZiYang-xie/WorldGen

    WorldGen is a tool that generates immersive 3D scenes from text prompts or images in seconds, supporting both Gaussian Splatting and mesh outputs for VR, games, and simulations.

  • Project
    Tencent-Hunyuan/HY-WorldPlay

    HY-World 1.5 is a real-time interactive world modeling framework that uses streaming video diffusion to generate geometrically consistent 3D environments based on user input.