MiniMax H3 Support in ComfyUI

ComfyUI now provides day-zero support for MiniMax H3, an open-weights omni-modal video model. This model allows users to generate high-resolution video up to 2K with native stereo audio, with clips lasting up to 15 seconds.

Model Capabilities and Multimodal Context

MiniMax H3 is designed as an omni-modal model, meaning it can process multiple input types—text, images, video, and audio—simultaneously to generate a final video output. This allows for a more integrated approach to video generation where the model resolves the relationship between different modalities based on a prompt.

Core Generation Modes

  • Text-to-Video (T2V): Generates video based solely on a text prompt.
  • Image-to-Video (I2V): Animates a static image into a video clip.
  • First-and-Last-Frame: Provides control over the opening and closing frames, allowing the model to interpolate the movement between them.
  • Reference-to-Video (R2V): Uses reference images, video, or audio to maintain consistency of a subject, motion, or voice throughout the clip.

Native Stereo Audio

Unlike many video models that add audio as a post-processing step, MiniMax H3 generates stereo audio natively within the same pass as the video generation. This ensures that the audio is a property of the model itself rather than a bolted-on addition.

Motion Transfer and Editing

The model supports motion transfer, where a reference video provides the camera movement, performance, or cutting rhythm, while the subject and style are defined by other inputs. This capability, combined with in-place editing, enables iterative refinement of specific shots.

Local Inference Optimization

To make MiniMax H3 accessible on consumer hardware, ComfyUI implemented several machine learning engineering optimizations to reduce the memory footprint.

Memory Reduction Techniques

  • Weight Pruning: Approximately 40% of the model's modulation weights were pruned and replaced with a functionally equivalent lookup table, which reduces memory usage without sacrificing output quality.
  • Quantization: The weights are shipped with int8 convrot quantization.
  • VRAM Management: Custom kernels and dynamic VRAM offloading are used to reduce peak VRAM usage during inference.

These optimizations reduced the total memory footprint from 123.6 GB in full precision to 42.5 GB for the smallest model variants. This allows the model to run locally on GPUs such as the RTX 3060.

User Implementation and Performance

To use MiniMax H3 in ComfyUI, users must update to version 0.30.0. Workflows for T2V, I2V, and R2V are available in the template library.

Real-World Performance Observations

Community members reporting their hardware setups have noted the following:

  • RTX 4070 Ti Super (16GB VRAM): A 10-second 480p video takes approximately 10 minutes to generate.
  • RTX GPU (General): A 10-second, half-megapixel video can be generated in a few minutes.

Community Insights

Users have highlighted the strengths and weaknesses of the model's current state:

"The mouse render is surprisingly good... The only one that looks 'off' is the beverage ad video during the can opening clip, it still has that 'AI smoothening' effect."

"Reference-to-video mode seems like all that was missing to enable completely independent cinematography as right now one couldn't stitch different scenes together properly without altering substantial portions of the scene."

Some users noted that while the model is excelent for standard scenarios, "jank" still occurs when attempting weirder, more complex concepts, such as a person being spun on a wheel.

Sources