ModelTC/LightX2V

LightX2V – A Lightweight Inference Engine for Image & Video Generation

What it is – LightX2V is an open‑source inference framework that lets you run a wide range of modern generative models (text‑to‑video, image‑to‑video, text‑to‑image, image‑to‑image, etc.) efficiently on a single GPU or a multi‑GPU cluster. It focuses on speed: the authors provide quantized, distilled LoRA checkpoints and a set of optimisations (off‑loading, tensor‑parallelism, FP8/NVFP4 quantisation, step‑distillation) that make diffusion‑style video generation several times faster than competing toolkits such as Diffusers, FastVideo or SGL‑Diffusion.

Core ideas

X2V – “X to Vision”: any input modality (text, image, audio) can be transformed into visual output (image or video). The framework abstracts this as a single LightX2VPipeline class that you configure with a model name, variant and a JSON config describing the inference schedule.
Unified support – The same pipeline works for many models: MiniMax‑H3 (omni‑modal video+stereo audio), LTX‑2/2.3, HunyuanVideo‑1.5, Wan‑2.1/2.2, Qwen‑Image, SwiftVR, etc.
Speed‑up tricks –
- Step‑distilled LoRAs (4‑step or 8‑step) that cut the number of diffusion steps dramatically while preserving quality.
- Quantisation (FP8, NVFP4) and tensor/sequence parallelism to fit large models on modest hardware.
- Disaggregated deployment – support for specialised accelerators (T‑head PPU, iluvatar, Intel AIPC PTL, Cambricon MLU, MUSA, AMD ROCm, Ascend, Hygon DCU, Enflame GCU, etc.) via custom kernels.
Extensible – Users can add new models by providing a checkpoint, a small config file and (optionally) a custom LoRA.

Quick‑start example (MiniMax‑H3 T2AV)

from lightx2v import LightX2VPipeline

pipe = LightX2VPipeline(
    model_path="/path/to/MiniMax-H3",
    model_cls="minimax_h3",
    model_variant="fl2av",
)
pipe.create_generator(config_json="configs/minimax_h3/dmd/minimax_h3_bf16_4step.json")

pipe.generate(
    task="t2av",
    seed=42,
    prompt="A cinematic fox walks through a snowy forest while soft wind and distant birds create an immersive winter soundscape.",
    save_result_path="outputs/minimax_h3_t2av_768p.mp4",
)

The snippet shows the typical workflow: instantiate a pipeline, load a distilled LoRA config, then call generate with a task name and a prompt.

Performance highlights (as reported by the authors)

Hardware	GPUs	Step time (s)	Relative speedup vs Diffusers
H100	1	5.18	1.9×
H100	8	0.75	3.9×
RTX‑4090D	1	20.26	1.5×
RTX‑4090D	8	4.75	3.3×
H100 8‑GPU, FP8, no‑CFG	0.35	2.1×
RTX‑4090D 8‑GPU, FP8, no‑CFG	2.35	2.0×
These numbers are measured on a 40‑step, 81‑frame video (Wan2.1‑I2V‑14B‑480P). The framework also offers 4‑step distilled models that can run in a fraction of the time of a typical 50‑step diffusion run.

Getting started

Docker (recommended) – Pull the official image from Docker Hub (lightx2v/lightx2v) and follow the Quick‑Start section in the docs.
From source – pip install -v git+https://github.com/ModelTC/LightX2V.git or clone the repo and run uv pip install -v ..
Optional operators – For the custom attention kernels and quantisation ops, see the “Install Attention/Quantize Operators” page in the docs.
Examples – The repo ships a rich examples/ folder (MiniMax‑H3, Wan, SwiftVR, HunyuanVideo‑1.5, etc.) and a scripts/ directory with ready‑to‑run inference scripts.

Ecosystem & Community

Model hub – All supported models and LoRAs are hosted on a dedicated HuggingFace organization (lightx2v). Users can pull the distilled checkpoints directly from there.
Online demo – A free web UI (LightX2V Studio) lets you try the framework without installing anything.
Mac app – A native macOS inference app (LightX2V‑Mac‑App) provides local video generation on Apple silicon.
Contributions – The project lists a set of community contributors and provides a pre‑commit style guide for code formatting.
Support channels – WeChat group (Robot ID: random42seed) and issue tracker on GitHub.

Who should use it?

Researchers needing fast inference for video diffusion models on a single or multi‑GPU server.
Developers building AI‑powered video generation products who want a ready‑made pipeline and pre‑optimised LoRAs.
Hobbyists who want to run text‑to‑video locally (e.g., on a MacBook) without massive GPU memory.

Bottom line

LightX2V is a purpose‑built, performance‑focused inference framework that unifies many state‑of‑the‑art generative models under a single API. Its emphasis on quantisation, step‑distillation, and broad hardware support makes it a practical choice for anyone looking to generate images or videos quickly and at scale.