dc-ai-projects/DC-Gen

What it solves

DC-Gen addresses the low throughput and high computational costs associated with modern high-resolution visual diffusion models. It specifically solves the problem of prohibitive training and inference costs for native 4K image generation and the inefficiency of having to retrain diffusion models from scratch whenever an autoencoder is changed.

How it works

DC-Gen is an acceleration framework that transfers a pre-trained diffusion model into a deeply compressed latent space using lightweight post-training. It employs a process called "Embedding Alignment" to transfer the base model's knowledge to the new latent space, allowing the model to maintain correct semantics without needing to fine-tune the diffusion model's weights initially. Quality is then fully recovered through LoRA fine-tuning. This approach reduces token redundancy, enabling significantly faster inference and native super-resolution capabilities.

Who it’s for

This project is for researchers and developers working with diffusion models who need to generate high-resolution images (up to 4K) or videos more efficiently without sacrificing the visual quality or text-rendering capabilities of their base models.

Highlights

• Dramatic Speedups: Achieves up to 53.8x faster inference on H100 GPUs at 4K resolution.
• Versatile Application: Validated across text-to-image, text-to-video, image-to-video, and image editing tasks.
• Native 4K Support: Enables native super-resolution image generation for models like FLUX and Z-Image that previously found it cost-prohibitive.
• Efficient Adaptation: Uses Embedding Alignment and LoRA fine-tuning to adapt to new autoencoders without full retraining.