yoloytang/Awesome-Video-LMM-Post-Training

🔥🔥🔥 Latest Papers, Codes and Datasets on Video-LMM Post-Training

What it solves

This project addresses the challenge of improving how Large Multimodal Models (LMMs) reason about video content. It provides a curated, systematic resource for researchers to track advancements in post-training techniques that help these models move beyond simple description toward complex, multi-step logical reasoning.

How it works

The repository organizes the latest research into four primary categories:

  • Reinforced Video-LMMs: Methods that use reinforcement learning (like RLHF or DPO) to align models with human preferences and improve factual accuracy.
  • SFT for Reasoning: Techniques using Supervised Fine-Tuning on reasoning-centric datasets, often incorporating Chain-of-Thought (CoT) to teach models how to think through video tasks.
  • Test-Time Scaling: Strategies that boost reasoning performance during use (inference) without further training, such as agentic frameworks, tool use, and RAG.
  • Benchmarks: A collection of challenging datasets specifically designed to measure and evaluate these advanced reasoning capabilities.

Who it’s for

This resource is designed for AI researchers, machine learning engineers, and developers working on video understanding, multimodal AI, or model alignment who want to stay current with the rapidly evolving landscape of video-LMM post-training.

Highlights

  • A comprehensive, categorized list of academic papers, code repositories, and datasets.
  • A focus on the latest research trends, including R1-style reasoning and process-based rewards.
  • A companion survey paper that provides a deep dive into the taxonomy and state-of-the-art in video reasoning.
  • An open-source, community-driven repository that welcomes contributions via pull requests.

Related

  • Project
  • Project
  • Project
  • Project