Immersive 3D Gaussian Splatting Reconstruction of Grace Cathedral

Immersive 3D Gaussian Splatting Reconstruction of Grace Cathedral

Overview

Vincent Woo has developed an immersive 3D tour of San Francisco's Grace Cathedral, demonstrating the power of 3D Gaussian Splatting (3DGS) for high-fidelity digital preservation of historic architecture. The project utilizes a combination of photographic capture, advanced reconstruction algorithms, and a web-based rendering engine to create a photorealistic reconstruction of the cathedral and its immediate surroundings.

Technical Implementation

3D Gaussian Splatting (3DGS) Workflow

Based on the project's history and similar work by the author, the reconstruction process follows a 3D Gaussian Splatting workflow. This emerging technology allows for the creation of highly detailed 3D models from a set of photographs. The typical pipeline involves:

  1. Data Collection: Capturing thousands of photographs from multiple angles, often utilizing drones for aerial perspectives.
  2. Alignment: Using software such as RealityCapture to align images, teaching the computer to identify corresponding points across different photographs.
  3. Splat Generation: Using tools like gsplat to produce the final 3D model (or "splat").

Rendering and Integration

The experience is hosted on the web and built on the PlayCanvas engine, with implementation support from Donovan Hutchence. The project also received early prototyping support from Isaac Mason and Andreas Sundquist of World Labs.

Key Features and User Experience

Visual Fidelity and Environmental Detail

The reconstruction captures a high level of detail, including the incidental environmental details such as nearby apartment towers, basketball courts, and solar panels. Users noted that the 3DGS approach is particularly effective at capturing reflections and the complex lighting of stained glass, which are traditionally difficult to render in traditional 3D modeling.

Immersive Elements

To enhance the immersion, the author integrated animated moving cars and sound effects, which users noted significantly increased the realism of the scene.

The "Peek" Mode

The project includes a "Peek" feature, described by users as a "sci-fi version of an exploded view," allowing users to explore the internal structure of the cathedral in a manner similar to architectural cross-sections.

Community Feedback and Analysis

Technical Challenges

Some users reported technical issues on specific browser/hardware combinations. For example, a user on Firefox/Windows 11 with an Nvidia GPU reported WebGPU errors related to max_buffer_binding_size limits, suggesting that the project pushes the boundaries of current web-based GPU acceleration.

User Experience Design

While the most users praised the fidelity, some criticized the "on-rails" presentation style, arguing that it limits the exploration of specific details, such as the Keith Haring altarpiece, making them certain elements difficult to access.

Future Directions for 3DGS

Community members discussed the following potential enhancements for 3DGS technology:

  • Geometry Optimization: Converting flat surfaces into traditional rigid textured geometry to optimize performance.
  • Lighting Decoupling: Factoring the lighting away from the geometry to allow for dynamic lighting changes.
  • Lighting AI Integration: Using generative AI to fill in missing details or remove transient variables like pedestrians from the scene.

"The technology is in such an early stage and it already slaps this hard. Wonder what the next steps are for this tech?"

Sources