Eyeline-Labs/Vista4D

What it solves

Vista4D addresses the challenge of "video reshooting," which is the process of synthesizing a dynamic scene from a source video but from entirely new camera trajectories and viewpoints. It specifically solves the problem of distribution shifts and artifacts that occur when using imprecise 4D reconstructions of real-world videos to guide video generation.

How it works

Vista4D uses 4D point clouds to represent dynamic scenes. It bridges the gap between training and inference by training on noisy, reconstructed multiview videos, making the model robust to the artifacts typical of real-world 4D reconstructions. The framework incorporates temporally-persistent static points to preserve scene content and improve camera control. It is built as a finetuned version of the Wan 2.1 (T2V-14B) model.

Who it’s for

This tool is designed for researchers and developers in computer vision and video synthesis who need to change camera angles in existing videos, expand dynamic scenes, or perform 4D scene recomposition.

Highlights

• Robust 4D Guidance: Capable of handling imprecise 4D reconstructions from real-world videos.
• Flexible Applications: Supports dynamic scene expansion, 4D scene recomposition (point cloud editing), and long video inference with memory.
• Interactive Camera UI: Includes a Viser-based UI for designing target camera paths and editing point clouds.
• Multi-GPU Support: Implements Unified Sequence Parallel (USP) to reduce VRAM usage and speed up inference.