Immersive 3D Reconstruction of Grace Cathedral via Gaussian Splatting

High-Fidelity Digital Preservation of Grace Cathedral

Vincent Woo has developed an immersive 3D tour of Grace Cathedral in San Francisco, utilizing 3D Gaussian Splatting (3DGS) to create a highly detailed digital reconstruction of the historic site. The project, built on the PlayCanvas engine, demonstrates the current capabilities of neural rendering for digital preservation, capturing complex architectural details, stained glass, and the surrounding urban environment.

Technical Implementation and Workflow

While the specific workflow for the Grace Cathedral project was not detailed in the primary source, the author's previous work on the Sutro Tower reconstruction provides a blueprint for the process used in 3D Gaussian Splatting projects:

  1. Data Collection: Capturing thousands of photographs of the subject, often utilizing drones for aerial perspectives.
  2. Alignment: Using software such as RealityCapture to align images, ensuring the computer identifies corresponding points across different photographs.
  3. Model Generation: Utilizing tools like gsplat to produce the final 3D Gaussian Splat model.

For the Grace Cathedral tour, the implementation includes a "Peek" feature, which provides a cutaway view of the architecture, and integrated audio and animated elements (such as moving cars) to increase immersion.

Key Capabilities of 3D Gaussian Splatting

3D Gaussian Splatting offers several advantages over traditional photogrammetry and surface texture mapping:

  • Handling Complex Materials: The technology is particularly effective at capturing reflections and the translucent quality of stained glass.
  • Mobile Performance: 3DGS allows for high levels of detail to be rendered on mobile hardware specifications.
  • Environmental Context: The reconstruction captures not only the primary structure but also incidental details like nearby trees, basketball courts, and adjacent apartment buildings, creating a comprehensive spatial record.

Technical Limitations and Observations

Despite the high fidelity, community analysis of the model highlights several inherent challenges and limitations of the current technology:

  • Artifacts and Rendering: Some users observed "stretched, hazy ellipses" in complex patterns, such as marble floors and certain paintings, which are characteristic of the Gaussian "splats" used as building blocks.
  • Hardware Constraints: Some users reported WebGPU errors on specific browser/hardware combinations (e.g., Firefox on Windows 11 with Nvidia GPUs), specifically regarding max_buffer_binding_size limits.
  • Lighting and Realism: Some viewers noted that the lighting can occasionally feel "off," suggesting a gap between neural rendering and absolute physical realism.
  • Navigation: The "on-rails" presentation style was noted by some as a limitation compared to free-roaming exploration.

Future Directions for Neural Reconstruction

Discussion around the project suggests several potential evolutions for 3D reconstruction technology:

  • Hybrid Geometry: Converting flat surfaces into classic rigid textured geometry to optimize performance and visual clarity.
  • AI-Driven Optimization: Using segmentation to automatically remove transient objects (like humans) and generative AI to fill in missing non-semantic details.
  • Lighting Decoupling: Factoring lighting away from the geometry to allow for dynamic lighting changes within the reconstructed scene.

Sources

Related

  • Project
    Linketic/CityGaussian

    A framework for high-quality, large-scale 3D scene reconstruction using Gaussian Splatting, featuring multi-GPU support and geometrically accurate rendering.

  • Dispatch
    Capturing Hyper-Realism: The Art and Science of Gaussian Splatting a Strawberry

    An exploration of a high-fidelity 3D reconstruction of a strawberry using Gaussian Splatting, detailing the rigorous photographic setup and the technical discourse surrounding the technology.

  • Dispatch
    Introduction to 3D Gaussian Splatting

    3D Gaussian Splatting is a rasterization technique that enables real-time rendering of photorealistic 3D scenes learned from a small set of images.

  • Project
    zju3dv/InfiniSplat

    InfiniSplat is a system for large-baseline monocular view synthesis that reconstructs 3D Gaussian Splatting scenes from a single RGB image or an RGB-depth pair.

  • Project
    ArthurBrussee/brush

    A cross-platform 3D reconstruction engine using Gaussian splatting that runs on WebGPU and the Burn framework to support macOS, Windows, Linux, Android, and browsers.