Luisacaotica/ComfyUI-MiniMaxH3Mod

ComfyUI‑MiniMaxH3Mod – Reference‑mod support for MiniMax H3 in ComfyUI

What it is

• A custom node pack for the visual‑programming UI ComfyUI that adds a set of nodes for creating, loading, inspecting and applying “RefMods” – reusable reference bundles for the MiniMax H3 model (image, video and audio references).
• The nodes do not train MiniMax H3 or replace its diffusion core; they simply encode visual/audio data with the model’s VAE, store the latent tensors in .safetensors files, and feed those latents back to the model during generation.

Key concepts

Concept	Meaning
RefMod	A .safetensors file that contains a visual, video or audio latent together with metadata (token count, shape, etc.).
Create H3 RefMod	Encode a single image/video/audio into a RefMod.
Create H3 RefMod Master	Encode visual and audio together, producing a single bundle file (or two separate files).
Apply H3 RefMod	Combine one or more loaded RefMods with a generation prompt, supplying the stored latents as conditioning.
Load H3 RefMods / Load H3 RefMod Axis	UI widgets that browse the registered refmods folders and output the selected bundles.
Bundle	A single .safetensors that holds multiple modalities (visual + audio) in version‑5 format.

Main features (as of v0.2.6)

• Standalone creators – Create H3 RefMod and Create H3 RefMod Master can run without a downstream node; they return a details string and optionally write files.
• Mask list handling – Collect H3 RefMod Masks lets you feed a list of masks, each kept at its original resolution, to the creator nodes.
• Full‑reference video sampling – you can specify a causal frame count before sampling, avoiding the old 16‑frame hard limit.
• Compact additive loaders – Loader nodes start with one visible slot; click + Add RefMod to expose up to eight slots, each with its own All / Visual / Audio selector and strength controls.
• Single‑file visual + audio bundles – With save_layout=bundle the Master node writes one .safetensors that contains both visual and audio references.
• Library & inspector UI – Search/filter stored RefMods, preview first‑frame or full video, listen to a short audio preview, and see token‑budget information.
• Budget policies – truncate (default) automatically cuts excess tokens; error aborts if the token budget is exceeded. Separate max_tokens for visual, audio_max_tokens for audio, and an overall max_total_tokens limit.
• Refresh RefMods – Updates dropdown lists from the filesystem without resetting the whole ComfyUI model list.
• Cross‑platform support – Tested on Windows and Apple Silicon (M5 Ultra, macOS 27, PyTorch 2.14). Optional video back‑ends (opencv‑python, imageio).

Installation

# 1️⃣ Clone into ComfyUI's custom_nodes folder
git clone https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod \
    custom_nodes/ComfyUI-MiniMaxH3Mod
# 2️⃣ Restart ComfyUI (or reload custom nodes)
# 3️⃣ Dependencies are installed automatically by ComfyUI‑Manager or run:
pip install -r custom_nodes/ComfyUI-MiniMaxH3Mod/requirements.txt

The pack requires a working ComfyUI installation that already supports MiniMax H3 (i.e., the native H3 VAE and conditioning nodes). No extra MiniMax‑H3 code is bundled; the mod only talks to the existing VAE.

Typical workflow

1. Create a reference – Drop an image, a short video, or an audio clip into a Create H3 RefMod (or Master) node. Adjust max_tokens, budget_policy, mask/mask_list, etc. Press Run; a .safetensors file appears in the registered refmods folder.
2. Load the reference – Use Load H3 RefMods (or the Axis variant) to pick the saved file. You can load several slots at once and set per‑slot strengths.
3. Apply during generation – Connect the loader output to Apply H3 RefMod, then feed the resulting conditioning into the normal MiniMax H3 generation pipeline (e.g., a sampler node). The model will attend to the stored latents alongside the textual prompt.
4. Inspect / tweak – Open Inspect H3 RefMod to view token counts, preview the stored visual/audio, and adjust strengths before re‑running.

Limitations & known issues

• Voice cloning is not functional – Audio extraction works, but the current implementation cannot reliably transfer a speaker’s identity.
• No training – The “training” visual mode (training / Compressed Reference) is just a latent‑pooling + optional refinement step; it does not train LoRA or any other model.
• Token budget is hard – Exceeding max_tokens (visual) or audio_max_tokens aborts unless you choose truncate. The budget does not guarantee quality; it only caps attention cost.
• Audio‑visual sync not guaranteed – Bundles store visual and audio latents separately; they are not temporally aligned beyond what the source files provide.
• File‑system‑only persistence – RefMods are saved as .safetensors on disk; they are not version‑controlled by the node pack.
• Platform quirks – Tested primarily on Windows; Apple‑Silicon support exists but may not cover every workflow.

Where to start

• The repository ships a demo bundle: mods/vanellope_example.safetensors. After installing, it appears in the loader dropdown as vanellope_example. Plug it into Apply H3 RefMod at strength 1.0 and try a prompt like “a candy racer in a karting scene”.

Further reading

• Full changelog: CHANGELOG.md
• Validation & known limitations: REVIEW_FOLLOWUP.md
• Bundle file format details: BUNDLE_FORMAT.md

---

ComfyUI‑MiniMaxH3Mod is a storage‑ and conditioning‑focused extension for MiniMax H3. It lets artists capture visual/audio references once and reuse them efficiently in later generations, without retraining the model.