jayin92/Skyfall-GS

What it solves

Skyfall-GS addresses the difficulty of creating large-scale, geometrically accurate 3D urban scenes. Traditionally, this requires expensive 3D scans or annotations, which are scarce for generalizable models. This project enables the synthesis of immersive 3D cities using only readily available satellite imagery and open-domain diffusion models.

How it works

The framework uses a two-stage process to build 3D scenes:

1. Reconstruction: It first uses satellite imagery to establish the coarse 3D geometry of the urban area.
2. Synthesis with Iterative Dataset Update (IDU): It refines the geometry and adds photorealistic textures. This is achieved through a curriculum-driven strategy that uses a diffusion model to create high-quality close-up appearances, iteratively updating the dataset to improve visual consistency and detail.

The final output is a Gaussian Splatting model that can be rendered in real-time for immersive exploration.

Who it’s for

This tool is designed for researchers and developers working on 3D scene synthesis, urban planning visualization, and embodied AI applications that require realistic, large-scale virtual environments.

Highlights

• No 3D Annotations: Creates large-scale scenes without the need for costly manual 3D labels.
• Real-time Exploration: Supports immersive, real-time 3D navigation of the synthesized urban environments.
• Iterative Refinement: Uses a specific IDU strategy to progressively enhance both geometric completeness and texture photorealism.
• Cross-view Consistency: Improves geometric and textural consistency across different viewing angles compared to existing methods.