EzioBy/Ditto

[CVPR'26 Highlight] Ditto: Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset

What it solves

Ditto addresses the scarcity of high-quality data for instruction-based video editing. Existing models often struggle with limited scope and a trade‑off between computational cost and output quality, specifically regarding temporal coherence in edited videos.

How it works

Ditto uses a scalable pipeline to generate a synthetic dataset called Ditto-1M, consisting of one million high‑fidelity video editing examples. The process involves:

  1. Data Generation: Fusing a leading image editor with an in‑context video generator to create diverse editing pairs.
  2. Efficiency: Using a distilled model architecture and a temporal enhancer to reduce GPU overhead while maintaining temporal coherence.
  3. Agent‑Driven Control: An intelligent agent manages the pipeline by crafting diverse instructions and filtering outputs to ensure quality control.
  4. Model Training: A new model, Editto, is trained on this synthetic dataset using a curriculum learning strategy.

Who it’s for

Researchers and developers working on generative video AI, specifically those focused on instruction‑based video editing and synthetic data generation for training ML models.

Highlights

  • Ditto-1M Dataset: A massive synthetic dataset of 1 million high‑fidelity video editing examples.
  • Editto Model: A state‑of‑the‑art instruction‑following video editing model trained on the Ditto-1M dataset.
  • Flexible Deployment: Supports integration with DiffSynth and ComfyUI for inference.
  • Scalable Pipeline: An automated agent‑driven pipeline that invested over 12,000 GPU‑days to generate the training data.