minnie-lin/Awesome-Physics-Cognition-based-Video-Generation

A comprehensive list of papers investigating physical cognition in video generation, including papers, codes, and related websites.

What it solves

Keeping up with the fast-moving field of physics-cognition-based video generation is hard. This repository solves that by collecting and organizing the most relevant papers, code, and resources into a single, structured list.

How it works

The repository groups works into clear categories: basic schematic perception for generation, passive cognition of physical knowledge, active cognition for world simulation, datasets/benchmarks/metrics, and physical understanding (with sub-sections on VLMs/LLMs/MLLMs and benchmarks). Each entry links to its arXiv paper, code (if available), and project website. The repo also includes a companion survey paper and a table of contents for easy navigation. It is actively maintained and welcomes contributions via pull requests.

Who it's for

Researchers, students, and practitioners working on video generation, world models, and physics-aware AI. Anyone who wants a structured overview of how physical knowledge is being integrated into generative models will find this useful.

Highlights

  • Structured taxonomy of physics-cognition-based video generation research
  • Includes a survey paper with arXiv link
  • Covers T2V, V2V, and dynamic 3D/4D generation
  • Lists workshops, surveys, and detailed paper entries with code/website links
  • Actively updated with contribution guidelines

Related

  • Project
    Bomx/super-video-maker-skill

    An end-to-end AI video production skill for agentic coding frameworks that enables AI agents to create polished videos using avatars, b-roll, and automated composition.

  • Project
    SAOS-3624/dsh-video-storyboard

    A tool that summarizes long videos into a single storyboard image and a time-coded text list, enabling AI to analyze video content efficiently without the high token costs of frame-by-frame processing.

  • Project
    showlab/Kiwi-Edit

    Kiwi‑Edit is an open‑source video‑editing system that takes natural‑language instructions (and optional reference images) to modify whole videos. It fuses a multi‑modal LLM encoder with a video Diffusion Transformer, offering style transfer, object add/replace/remove, and background changes. The repo provides full training scripts (three‑stage curriculum with Qwen2.5‑VL‑3B + Wan2.2‑TI2V‑5B), pretrained Diffusers checkpoints, and evaluation pipelines on OpenVE‑Bench and RefVIE‑Bench.

  • Project
    openspline/openspline

    OpenSpline is a developer toolkit that adds synchronized, talking-head video avatars to existing voice AI agents, supporting both cloud and self-hosted GPU deployments.