QuanjianSong/FashionChameleon

Official Pytorch Code of the Paper "FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customization"

What it solves

FashionChameleon addresses the challenge of real-time, interactive human-garment customization in video generation. It allows for the switching of garments on a person in a video while maintaining high quality and consistency, achieving speeds significantly faster than previous customization methods.

How it works

Built upon the Wan2.2-TI2V-5B model, the framework employs three primary technical components:

  1. In-Context Learning Teacher: A teacher model trained using tailored image-to-video (I2V) priors and mismatched reference-garment pairs to handle garment switching.
  2. Streaming Distillation: Uses in-context learning and a Gradient-Reweighted DMD (Distribution Matching Distillation) approach. This method uses a reward model to dynamically reweight gradients during self-rolling, ensuring later frames in a sequence maintain quality and consistency.
  3. KV-Cache Rescheduling: A training-free mechanism that enables interactive multi-garment customization during the generation process without requiring specific training data of people switching clothes.

Who it’s for

This project is designed for researchers and developers working on AI-driven fashion, virtual try-on, and real-time video synthesis.

Highlights

  • Real-Time Performance: Generates 720p video at 23.8 FPS on a single H200 GPU.
  • Interactive Customization: Supports switching garments on the fly during video generation.
  • Improved Consistency: Uses gradient-reweighting to prevent quality degradation in extrapolated frames.
  • High Efficiency: Claims to be 30’s to 180’s faster than existing customization methods.

Related

  • Project
    jamichss/Stream-DiffVSR

    Stream‑DiffVSR is a diffusion‑based video‑super‑resolution system designed for low‑latency, online use. It runs a 4‑step distilled denoiser, an auto‑regressive temporal guidance module, and a lightweight temporal decoder to upscale 720p video frames in ~0.33 s on an RTX 4090, far faster than prior diffusion VSR methods. The repo provides conda environments, CLI and Gradio UI for inference, optional TensorRT acceleration, Docker support, and full training scripts using REDS and Vimeo‑90K datasets.

  • Project
    seesee75-commits/ComfyUI-MiniMaxH3-Director

    A timeline editor for MiniMax H3 in ComfyUI that replaces manual storyboard prompting with a visual interface for managing shots, keyframes, and audio.

  • Project
    a312863063/Video-Auto-Wipe

    A deep learning video inpainting project that automatically detects and erases fixed-pattern content like subtitles, static logos, and dynamic icons from videos. It uses a two-step training process (time-domain perception + end-to-end inpainting fine-tuning) with synthetic foreground data overlaid on real video backgrounds. The subtitle removal model is open-sourced, while logo removal models are demonstrated but not released.

  • Project
    ayushjain1144/trackeverything

    A CMU + Meta research project on dense point tracking in videos over long time horizons, using de-duplicated 3D scene representations. Paper is out; code is promised soon.