pollockjj/ComfyUI-MultiGPU

What it solves

ComfyUI-MultiGPU addresses the problem of limited GPU VRAM when running large AI models. It allows users to distribute model components (like the UNet, VAE, or CLIP) across multiple GPUs or offload them to system RAM (CPU DRAM), freeing up the primary compute GPU's VRAM for larger latent space processing, such as generating longer videos or higher-resolution images.

How it works

The project uses a system called DisTorch (Distributed Torch) to move static parts of a model off the main compute card. It provides several allocation modes:

• Normal Mode: Uses a simple slider to offload a specific amount of "virtual VRAM" to a single donor device.
• Expert Mode: Offers precise control via text strings for three different strategies:
  • Bytes: Specifies exact gigabytes or megabytes for each device (e.g., cuda:0,2.5gb;cpu,*).
  • Ratio: Distributes the model based on a percentage ratio (e.g., cuda:0,25%;cpu,75%).
  • Fraction: Allocates based on a fraction of each device's total available VRAM.

Who it’s for

This tool is for ComfyUI users who have multiple GPUs or limited VRAM on their primary card and want to run larger models (like FLUX, WanVideo, or GGUF-quantized models) without encountering out-of-memory errors.

Highlights

• Universal Support: Works with .safetensors and GGUF-quantized models.
• Extensive Node Library: Automatically creates MultiGPU versions of standard ComfyUI loaders and integrates with third-party extensions like WanVideoWrapper and Florence2.
• Flexible Offloading: Ability to offload the entire model to system RAM while still performing computation on the GPU.
• Detailed Documentation: Includes built-in help for over 70 nodes.