jayin92/Skyfall-GS

What it solves

Skyfall-GS addresses the difficulty of creating large-scale, geometrically accurate 3D urban scenes. Traditionally, this requires expensive 3D scans or annotations, which are rarely available at scale. This project enables the synthesis of immersive 3D city environments using only readily available satellite imagery and open-domain diffusion models.

How it works

The framework operates in two primary stages:

1. Reconstruction: It first uses satellite imagery to establish the coarse 3D geometry of the urban scene.
2. Synthesis with Iterative Dataset Update (IDU): It employs a curriculum-driven refinement strategy that uses diffusion models to generate high-quality, close-up textures and progressively improve the geometric completeness and photorealism of the scene.

The final output is a 3D model (provided as a fused PLY file) that allows for real-time, immersive exploration.

Who it’s for

This tool is designed for researchers and developers working on embodied AI applications, urban planning simulations, and immersive 3D environment generation.

Highlights

• No 3D Annotations: Creates large-scale scenes without the need for costly manual 3D labels.
• Hybrid Approach: Combines the structural accuracy of satellite data with the visual quality of generative diffusion models.
• Real-time Exploration: Supports immersive 3D navigation of the synthesized urban environments.
• Iterative Refinement: Uses a specific IDU strategy to ensure cross-view consistency and realistic textures.