kandinskylab/kandinsky-5
Kandinsky 5.0: A family of diffusion models for Video & Image generation
What it solves
Kandinsky 5.0 provides a family of high-quality diffusion models for generating videos and images from text prompts or input images. It specifically addresses the need for high-fidelity visual generation with strong support for both English and Russian languages and concepts.
How it works
The project utilizes a latent diffusion pipeline with Flow Matching. The core generative backbone is a Diffusion Transformer (DiT) that uses cross-attention to condition on text embeddings provided by Qwen2.5-VL and CLIP. For video processing, it employs the HunyuanVideo 3D VAE to encode and decode video into a latent space.
Who it’s for
- Researchers and enthusiasts who can use the pretrain models for further fine-tuning.
- Content creators looking for high-definition video (up to 10 seconds) and image generation.
- Developers seeking lightweight, efficient models (like the 2B Video Lite version) for faster inference.
Highlights
- Diverse Model Family: Includes "Pro" (19B parameters) for maximum quality and "Lite" (2B for video, 6B for image) for efficiency.
- Multimodal Capabilities: Supports Text-to-Video (T2V), Image-to-Video (I2V), and Text-to-Image (T2I).
- Optimized Performance: Offers CFG-distilled and Diffusion-distilled variants for significantly lower latency (up to 6 faster).
- Language Support: Strong understanding of Russian concepts alongside English.
- Controllable Motion: Includes LoRAs for camera control in video generation.