worldbench/DanceOPD

🔥 DanceOPD: On-Policy Generative Field Distillation

What it solves

DanceOPD addresses the problem of "capability interference" in image generation models. When developers try to combine multiple abilities—such as text-to-image generation, local editing, and global style transformations—into a single model, improving one often degrades another. DanceOPD provides a way to synthesize these diverse capabilities into one compact student model without the need for task-specific training code or complex reward models.

How it works

The system treats each desired capability as a "velocity field" provided by a frozen teacher model. During training, the student model performs a rollout (generating a state), and the system queries the corresponding teacher model for the correct velocity at that specific state. The student is then updated using a simple velocity-MSE objective to match the teacher's output.

Key technical features include:

  • On-policy field query: Teachers supervise states actually visited by the current student, rather than offline data.
  • Hard-routed matching: Each training sample is assigned to exactly one semantically valid teacher, preventing ambiguous averaging of different capabilities.
  • Semantic-side query: By default, it uses a single low-noise query state per rollout to maintain efficiency.
  • Backend extensibility: It supports multiple flow-matching backbones, including SD3.5 and Z-Image.

Who it’s for

This project is for AI researchers and developers working with flow-matching image generators who want to merge multiple specialized capabilities (like high-quality editing and prompt adherence) into a single deployed model.

Highlights

  • Capability Fusion: Successfully combines T2I, local editing, and global editing without collapsing toward any single source.
  • Efficient Rollouts: Includes a query_prefix mode that can reduce rollout forwards by up to 31.7%.
  • Simple Objective: Uses direct velocity MSE, eliminating the need for adversarial critics or reward models.
  • Flexible Training: Offers three methods: danceopd (single-query ODE), diffusionopd (dense ODE-KL), and flowopd (SDE + PPO).

Related

  • Project
    worldbench/DiffusionOPSD

    A reward-guided post-training framework for diffusion models that uses on-policy self-distillation to create explicit intermediate targets, improving image quality and training efficiency.

  • Project
    vvvvvjdy/D-OPSD

    D-OPSD is an on-policy self-distillation framework that allows step-distilled diffusion models to be continuously tuned for new styles and concepts while preserving their fast inference capabilities.

  • Project
    chrisliu298/awesome-on-policy-distillation

    A curated collection of papers, frameworks, and tools for on-policy distillation, a post-training technique that trains student LLMs on their own generated samples to bridge the train-inference distribution gap.

  • Dispatch
    Segmind SD-Small and SD-Tiny Knowledge Distillation Release

    Segmind open-sourced the code and pretrained weights for SD‑Small and SD‑Tiny, two diffusion models that are 35% and 55% smaller than Stable Diffusion while delivering up to 2× faster inference.

  • Project
    Thinking-Space/Rethinking-OPD

    A research project and toolkit for analyzing and improving On-Policy Distillation (OPD) in LLMs, providing strategies to recover failing distillation and understanding its token-level mechanisms.