LogosIntelligenceLab/4d-gaussian-splatting

[ICLR 2024] Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting

What it solves

Traditional 3D scene rendering can capture static scenes, but real-world scenes move: people walk, cameras pan, objects shift. This project tackles the harder problem of representing and rendering dynamic scenes in real time while keeping them photorealistic. It does this by treating space and time together as one unified 4D model, instead of splitting them into separate steps.

How it works

The core idea is to extend "Gaussian splatting" — a technique that represents a scene using millions of tiny, fuzzy 3D blobs (Gaussians) that can be rendered quickly — from 3D to 4D. Each Gaussian now also carries a time component, so it can move and deform as the scene changes. The paper describes "unbiased 4D Gaussian primitives" and a dedicated rendering pipeline that handles both the spatial and temporal dimensions coherently. This allows the same model to render a scene from any viewpoint at any moment, including views that were never captured directly.

The code is built on top of the well-known 3D Gaussian Splatting implementation, so it uses the same environment and training approach. You download a dataset (either the Neural 3D Video dataset or the DNeRF dataset), preprocess it if needed, then run a single training command with a config file.

Who it's for

Researchers and developers working on novel view synthesis, virtual reality, film production, or any application that needs to render live or recorded dynamic 3D scenes in real time. It assumes familiarity with 3D Gaussian splatting and standard deep learning tooling (Python, conda).

Highlights

  • Real-time rendering of dynamic scenes (not just static ones) with photorealistic quality
  • Unified 4D Gaussian primitives that model space and time together, avoiding separate motion estimation steps
  • Official implementation of a paper published at ICLR 2024
  • Built on the mature 3D Gaussian Splatting codebase, easing adoption
  • Includes demo videos showing dynamic novel view synthesis, bullet-time effects, free-viewpoint video from a teleporting camera, and monocular dynamic scene reconstruction

Related

  • Project
  • Project
  • Project
  • Project