MiniMax H3 Support in ComfyUI

ComfyUI has implemented day-zero support for MiniMax H3, an open-weights omni-modal video model that generates high-resolution video up to 2K with integrated stereo audio. Through significant machine learning optimizations, the model is accessible on consumer-grade GPUs, including the RTX 3060.

Core Capabilities of MiniMax H3

MiniMax H3 is a third-generation video model that collapses multiple generative tasks into a single omni-modal framework. It can process text, images, video, and audio inputs to produce clips up to 15 seconds in length.

Multimodal Context Understanding

The model's primary strength is its ability to resolve relationships between different input modalities. Users can provide a combination of images, audio, and video along with a prompt describing how these elements relate, and the model handles the cross-modal synthesis automatically.

Native Stereo Audio

Unlike many video models that append audio as a post-process, MiniMax H3 generates stereo audio natively within the same pass as the video generation. This ensures that the sound is a fundamental property of the generated scene rather than an overlay.

Generation Modes

Text-to-Video: Generates video based solely on text prompts.
Image-to-Video: Animates a static image into a video clip.
First-and-Last-Frame: Allows users to define the starting and ending frames, with the model interpolating the motion between them.
Reference-to-Video: Uses reference images, video, or audio to maintain consistency of a subject, motion, or voice across the clip. This is particularly useful for motion transfer, where a reference video provides the camera movement or performance while the subject and style are defined separately.

Local Inference Optimizations in ComfyUI

To enable MiniMax H3 to run on consumer hardware, ComfyUI applied several engineering optimizations to reduce the model's massive memory requirements.

Weight Pruning and Lookup Tables

Engineers discovered that approximately 40% of the model's total parameters—specifically the modulation weights—could be pruned and replaced with a functionally equivalent lookup table. This reduction in weights was achieved without a loss in output quality.

Quantization and VRAM Management

ComfyUI utilizes int8 convrot quantization and custom kernels to lower peak VRAM usage. When combined with dynamic VRAM offloading, these optimizations reduced the total memory footprint by 66%, dropping from 123.6 GB in full precision to 42.5 GB for the smallest model variants.

Community Insights and Performance

Early adopters and community members have noted a significant leap in quality compared to previous state-of-the-art (SOTA) models, though some limitations remain.

Hardware Performance

User reports indicate varying generation times based on hardware:

"Im running this on my 4070ti super (16 gb vram), and it takes 10 minutes for a 10-seconds 480p video. but the results are spectacular."

Other users reported that 10-second, half-megapixel videos can be generated in a few minutes, marking a speed improvement over earlier models like WAN.

Qualitative Feedback

While the "Reference-to-video" mode is praised for enabling more precise cinematography and scene stitching, some users noted that "AI smoothening" effects are still visible in certain close-up shots, such as the opening of a beverage can. Additionally, some users observed that the model can struggle with "weirder concepts" or non-standard scenarios, where the physics or logic of the scene may break down.

Implementation Guide

To use MiniMax H3 in ComfyUI, users must update to version 0.30.0 or use Comfy Cloud. The process involves:

Updating ComfyUI to v0.30.0.
Downloading specific workflows (T2V, I2V, or R2V) from the Comfy-Org template library.
Downloading the model weights from the Comfy-Org/MiniMax-H3 Hugging Face repository and placing them in the designated model directory.