ModelTC/LightX2V

Lightweight Image Video Action Generation Inference Framework

What it solves

LightX2V is a high-performance inference framework designed to make image and video generation more efficient. It addresses the high computational cost and memory requirements of state-of-the-art diffusion and autoregressive models, enabling faster synthesis and deployment on a wider range of hardware, including consumer GPUs.

How it works

The framework provides a unified platform for various generation tasks (text-to-video, image-to-video, text-to-image, and image-editing). It achieves speedups through several optimization techniques:

  • Step Distillation: Reducing the number of inference steps (e.g., from 50 down to 4) to drastically cut generation time.
  • Quantization: Supporting FP8 and NVFP4 formats to reduce memory footprint and increase throughput.
  • Memory Management: Implementing block-level and phase-level offloading to allow large models to run on GPUs with lower VRAM.
  • Parallelism: Utilizing CFG and Ulysses parallelism to distribute workloads across multiple GPUs.
  • Hardware Support: Optimized for a variety of accelerators including NVIDIA (H100, RTX 4090), AMD ROCm, Ascend 910B, and others.

Who it’s for

  • AI Developers: Who need a unified, optimized pipeline for deploying multiple vision generation models.
  • Researchers: Looking to benchmark or deploy distilled and quantized versions of large video models.
  • Content Creators: Who want to generate high-quality AI video/images faster and with less hardware overhead.

Highlights

  • Massive Speedups: Claims up to 20x acceleration on single GPUs via distillation and system optimizations.
  • Broad Model Support: Compatible with LTX-2, HunyuanVideo-1.5, Wan2.1/2.2, and Qwen-Image.
  • Flexible Deployment: Offers Gradio, ComfyUI, and a Windows one-click installer for different user levels.
  • Multi-Hardware Compatibility: Supports a wide array of backends including NVIDIA, AMD, Intel AIPC, and various specialized AI chips.