inspatio/querysplat

repo for QuerySplat

What it solves

QuerySplat addresses the challenge of predicting 3D Gaussian Splatting (3DGS) representations from images, specifically by decoupling the representation of geometry and appearance to improve the quality and accuracy of 3D scene reconstructions.

How it works

The system uses a feed-forward model to predict 3D Gaussians for rendering. It leverages VGGT-Omega for image encoding, camera prediction, and depth prediction. The process involves custom-image preprocessing and can be enhanced with optional Test-Time Optimization (TTO) to refine the reconstruction objective using LPIPS weights. It can operate in a "pose-free" mode where it estimates its own camera parameters rather than relying on provided dataset poses.

Who it’s for

This project is intended for researchers and developers working on 3D scene reconstruction, neural rendering, and computer vision, particularly those interested in efficient 3D Gaussian Splatting prediction.

Highlights

  • Decoupled Representations: Separates geometry and appearance for better 3DGS prediction.
  • Pose-Free Capability: Can estimate camera parameters internally without requiring external pose data.
  • Test-Time Optimization: Includes an optional TTO step to further refine scene quality.
  • Integration: Built upon VGGT-Omega for robust image encoding and depth/camera prediction.

Related

  • Project
    cvg/resplat

    ReSplat is a feed-forward recurrent model for 3D Gaussian splatting that iteratively refines scene representations using rendering error as feedback.

  • Project
    Linketic/CityGaussian

    A framework for high-quality, large-scale 3D scene reconstruction using Gaussian Splatting, featuring multi-GPU support and geometrically accurate rendering.

  • Dispatch
    Introduction to 3D Gaussian Splatting

    3D Gaussian Splatting is a rasterization technique that enables real-time rendering of photorealistic 3D scenes learned from a small set of images.

  • Project
    zju3dv/InfiniSplat

    A feed-forward 3D Gaussian reconstruction framework that generates 3D scenes from either a single RGB image or multiple overlapping images for novel-view rendering.

  • Project
    caiyuanhao1998/Open-DiffusionGS

    A single-stage image-to-3D generation and reconstruction framework that integrates Gaussian Splatting into a diffusion denoiser for fast, scalable 3D creation.