zju3dv/4K4D

What it solves

4K4D enables real-time 4D view synthesis at 4K resolution, allowing users to render high-resolution dynamic scenes from novel viewpoints in real-time.

How it works

The project utilizes a volumetric video framework called EasyVolcap. It employs a point-based representation and supports a tile-based CUDA renderer for efficient rasterization. The system can be trained on multi-view datasets (such as DNA-Rendering or ZJU-MoCap) and can be optimized for different targets, including a real-time rendering model and a mobile-optimized version that distills appearance models into low-degree spherical harmonics (SH) parameters.

Who it’s for

Researchers and developers working in computer vision, 4D reconstruction, and volumetric video who need high-fidelity, real-time playback of dynamic scenes.

Highlights

• 4K Resolution: Supports high-resolution synthesis of dynamic scenes.
• Real-Time Performance: Optimized for real-time rendering, including a specialized CUDA-backend tile rasterizer.
• Flexible Rendering: Includes a WebSocket-based viewer for remote server-side rendering with local client manipulation.
• Modular Framework: Built on the EasyVolcap framework, allowing for easy integration of training and inference pipelines.