WorldGen: a fast text-to-3D scene generator for creating immersive 360° environments

What it solves

WorldGen allows users to create complete 3D scenes from simple text prompts or 2D images in a matter of seconds. It eliminates the need for manual 3D modeling, enabling the rapid generation of immersive environments for games, simulations, robotics, and virtual reality.

How it works

WorldGen generates 3D scenes by leveraging a pipeline that can process text or images into 3D representations. It internally supports generating scenes from 360° panorama images (equirectangular) and utilizes tools like DA-2 for 360° depth estimation and FLUX.1-dev for image generation. The system can output the scene as either Gaussian Splatting (.ply) or a 3D mesh, allowing for flexible rendering and real-time exploration.

Who it’s for

This tool is designed for developers and creators in the gaming, VR/AR, and robotics industries who need to quickly prototype 3D environments or generate synthetic data for simulations.

Highlights

• Instant Generation: Creates full 3D scenes from text or images in seconds.
• 360° Exploration: Supports consistent, free-viewpoint exploration with loop closure.
• Versatile Scene Support: Works for both indoor and outdoor environments, including realistic and stylized scenes.
• Flexible Output: Supports both Gaussian Splatting and mesh generation for different rendering needs.
• Low VRAM Mode: Includes a mode for GPUs with less than 24GB VRAM (using approximately 10GB).
• Real-time Rendering: Supports rendering at any resolution with custom camera trajectories.