thu-ml/DiT-Extrapolation
Official implementation for "RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers" (ICML 2025) , UltraViCo (ICLR 2026) and UltraImage
What it solves
이 프로젝트는 Diffusion Transformers (DiT)가 학습된 길이 또는 해상도를 초과하는 콘텐츠를 생성할 때 발생하는 제한 사항을 해결합니다. 구체적으로, "외삽(extrapolation)"을 가능하게 하여, 사전 학습된 모델이 대규모 재학습 없이도 더 긴 비디오(예: 5s 비디오를 11s로 확장)와 더 높은 해상도의 이미지를 생성할 수 있도록 합니다.
How it works
핵심 기술인 RIFLEx는 1D Rotary Positional Embedding (RoPE)의 특정 구성 요소의 고유 주파수를 줄여, 생성된 콘텐츠가 외삽 후에도 단일 주기 내에 머물도록 합니다. 이는 원래의 RoPE 구현에 단 한 줄의 코드를 추가함으로써 달성할 수 있으며, 플러그 앤 플레이 방식의 솔루션입니다. 또한, 이 저장소에는 비디오 및 이미지 생성의 외삽 한계를 더욱 확장하기 위한 UltraViCo 및 UltraImage 구현이 포함되어 있습니다.
Who it’s for
HunyuanVideo, CogVideoX, Wan2.1, Flux, 그리고 Qwen-Image와 같은 기존 SOTA 모델의 출력 길이 또는 해상도를 높이고자 하는 비디오 및 이미지 확산 모델 연구자 및 개발자를 대상으로 합니다.
Highlights
- Plug-and-Play: 최소한의 코드 변경으로 사전 학습된 모델에 적용할 수 있습니다.
- Temporal Extrapolation: 비디오 길이를 확장합니다 (예: HunyuanVideo를 5s에서 11s로, CogVideoX를 6s에서 12s로).
- Broad Model Support: 여러 SOTA 아키텍처를 포함하여 HunyuanVideo, CogVideoX, Wan2.1, Flux, 그리고 Qwen-Image와 호환됩니다.
- Training-Free Option: 다양한 성능 요구 사항에 따라 학습이 필요 없는 버전과 미세 조정(fine-tuned) 버전을 모두 제공합니다.
관련
- 프로젝트wildminder/ComfyUI-DyPE
FLUX 및 Qwen Image와 같은 Diffusion Transformer를 위해 학습 없이 모델 패칭(model patching)과 캐스케이드 정제(cascade refinement)를 사용하여 초고해解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解text_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________- 4K+ 생성 기능을 제공하는 ComfyUI 커스텀 노드 팩입니다. 4K+ 생성 기능을 제공하는 ComFLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLX, FLUX, FLUX, content_content_content_content_content_content_content_content_content_content_content_content_content__content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_}^{\ even more detailed instructions for instructions-based instruction-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-tuning-content_content_content_content_content_________________________________ _content_content_content_rectify-flow models, 使用하여 초고해解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解解_content_content__content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_cup_content_content_content_content_content_content_content_content_content_content_content_content_content _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ _________________________________________________________________________________________________________________________________________________________________ _________________________________________________________________________________________________________________________________ _________________________________________________________________________________ ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________解解解解解解解解解解解解解解解解解解解解解解解解解解解________________________________________________________________________________________________________________________________________________________________________________________________________________ ________________________________________________________________________________________________________________________________________________________________________________________________________________text_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_content_________________________________________________________________________________________________________________________________________________________________ _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________- 4K+ 생성 기능을 제공하는 ComfyUI 커스텀 노드 팩입니다. 4K+ 생성 기능을 제공하는 ComFLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FLUX, FL_content_content_content_content_content_content_content_content_content_content_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________* 4K+ 생성 기능을 제공하는 ComfyUI 커스텀 노드 팩입니다. FLUX, Qwen Image, Z-Image, Anima/Cosmos와 같은 Diffusion Transformer 모델을 사용하여 초고해상도 4K+ 이미지를 생성하거나 기존의 latents를 고해상도로 업스케일링하고 싶은 ComfyUI 사용자들을 위한 것입니다. ## Highlights * **Multi-Architecture Support:** FLUX, Nunchaku, Qwen Image, Krea-2, Z-Image, Anima/Cosmos와 호환됩니다. * **Training-Free:** 재학습이나 미세 조정이 필요하지 않습니다. 로더(loader) 이후에 모델을 패칭하는 방식으로 작동합니다. * **Ultra-High Resolution:** 4096×4096 이상의 해상도로 이미지를 생성할 수 있습니다. * **Performance Boosts:** 추론 시간을 단축하기 위해 희소 어텐션(sparse-attention) 가속을 위한 HAP를 포함합니다. * **Seamless Integration:** 기존 샘플러와 최적화 노드와 호환성을 유지하는 단일 노드 통합 방식입니다.
- 프로젝트Jasonzzt/ComfyUI-CacheDiT
지능적 캐싱을 통해 Diffusion Transformer (DiT) 모델을 가속화하는 ComfyUI 확장 기능으로, 이미지 및 동영상 생성 시 1.4x에서 2.0x의 속도 향상을 제공합니다.
- 프로젝트aigc-apps/VideoX-Fun
텍스트, 이미지, 동영상에서 동영상 생성을 지원하는 CogVideoX-Fun 및 Wan 2.1 모델을 활용하는 동영상 생성 파이프라인. 고도의 구조적 제어 및 카메라 제어 기능을 제공.
- DispatchDiffusers를 이용한 Stable Diffusion
Hugging Face는 Diffusers 라이브러리를 사용해 텍스트‑이미지 Stable Diffusion 모델을 실행하는 방법을 발표했으며, 코드, 라이선스 세부 정보 및 모델의 잠재 확산 아키텍처에 대한 설명을 제공합니다.
- DispatchDreambooth Stable Diffusion 파인‑튜닝 가이드 with Diffusers
Hugging Face는 Diffusers 라이브러리를 사용한 Dreambooth로 Stable Diffusion을 학습하기 위한 상세 권장 사항을 발표했으며, 낮은 학습률, 충분한 단계, 얼굴에 대한 prior preservation, 텍스트‑인코더 파인‑튜닝이 가장 높은 품질의 결과를 제공한다는 것을 보여줍니다.