kandinskylab/kandinsky-5
Kandinsky 5.0: A family of diffusion models for Video & Image generation
What it solves
Kandinsky 5.0 提供一個高品質擴散模型家族，用於從文字提示或輸入圖像生成影片與圖像。它特別針對高保真視覺生成的需求，並提供對英語與俄語兩種語言及概念的強大支援。
How it works
此專案使用帶有 Flow Matching 的潛在擴散管線。核心生成骨幹是使用交叉注意力（cross‑attention）以 Qwen2.5‑VL 與 CLIP 提供的文字嵌入作為條件的 Diffusion Transformer（DiT）。在影片處理方面，則採用 HunyuanVideo 3D VAE 將影片編碼與解碼至潛在空間。
Who it’s for
- Researchers and enthusiasts who can use the pretrain models for further fine-tuning.
- Content creators looking for high-definition video (up to 10 seconds) and image generation.
- Developers seeking lightweight, efficient models (like the 2B Video Lite version) for faster inference.
Highlights
- Diverse Model Family: Includes "Pro" (19B parameters) for maximum quality and "Lite" (2B for video, 6B for image) for efficiency.
- Multimodal Capabilities: Supports Text-to-Video (T2V), Image-to-Video (I2V), and Text-to-Image (T2I).
- Optimized Performance: Offers CFG-distilled and Diffusion-distilled variants for significantly lower latency (up to 6 faster).
- Language Support: Strong understanding of Russian concepts alongside English.
- Controllable Motion: Includes LoRAs for camera control in video generation.