chengzeyi/Comfy-WaveSpeed

What it solves

Comfy-WaveSpeed provides inference optimization for ComfyUI, reducing the time and computational cost required to generate images and videos using large diffusion models. It addresses the slow speed of local inference for high-resolution or complex models like FLUX, SD3.5, and HunyuanVideo.

How it works

The project implements two primary optimization techniques:

1. First Block Cache (FBCache): This dynamic caching algorithm uses the residual output of the first transformer block as an indicator. If the difference between the current and previous residual output is small enough, the system reuses the previous final residual output and skips the computation of all subsequent transformer blocks, potentially speeding up inference by 1.5x to 3.0x.
2. Enhanced torch.compile: An improved version of the standard PyTorch compilation process that optimizes the model's execution graph. Unlike the original TorchCompileModel node, this version is compatible with LoRA.

Who it’s for

Users of ComfyUI who run diffusion models locally on their own GPUs and want to increase generation speed without significant loss in image or video quality.

Highlights

• Broad Model Support: Works with FLUX, LTXV, HunyuanVideo, SD3.5, and SDXL.
• Significant Speedups: FBCache can achieve up to 2x speedup while maintaining high accuracy.
• LoRA Compatibility: The enhanced compilation node supports the use of LoRA models.
• Flexible Configuration: Users can adjust the residual_diff_threashold to balance speed and quality based on the specific model being used.