h3-metal: Native MiniMax-H3 Inference for Apple Silicon

h3-metal provides native MiniMax-H3 inference for Apple Silicon, implementing a sequence of optimized vertical slices including deterministic host/model metadata, portable Metal block parity, and end-to-end support for prompt-to-video/audio, first/last-frame conditioning, and ordered references. The project focuses on maximizing performance and memory efficiency on M3 Max and M5 Max hardware.

High-Performance Inference Presets

Users can balance generation speed and output quality through several independent controls. The default configuration is 20 denoising passes with 50 transformer blocks and a reuse factor of 1.

Speed and Quality Trade-offs

Control	Slow Reference	Default	Aggressive	Main Impact
Denoising Passes	--steps 50	--steps 20	--steps 4..7	Total number of actual denoising passes.
Whole Denoiser Reuse	--reuse 1	--reuse 2	--reuse 3	Reduces fresh DiT evaluations (e.g., 20 steps $\to$ 11 or 8).
Active DiT Blocks	--layers 50	--layers 45	--layers 40	Reduces compute and resident transformer weights.
Core Residual Reuse	--core-reuse 1	--core-reuse 4	--core-reuse 6	Refreshes patch/head work every step but runs expensive core less often.
Token Reduction	Off	Optional	--token-reduction	Pairs horizontal video tokens in middle blocks; faster but may affect composition.
Internal Canvas	Output Size	384x384	320x320	Runs DiT/VAE smaller, then upscales via vImage.

Performance Benchmarks

On an M5 Max, a four-pass denoise for a 512-square, 22-frame test took approximately 3.5 seconds, compared to 26.4 seconds for the 29-pass reference. Using token reduction on a 512-square canvas reduced denoise time from 16.69 to 12.60 seconds.

Advanced Conditioning and Media Support

h3-metal supports multiple conditioning paths to control video generation beyond simple text prompts.

Frame Anchoring and References

• FL2VA Path: Uses --first-frame and --last-frame to anchor the start and end of a video sequence.
• Ref2VA Path: Uses ordered references via --ref-image, --ref-silent-video, and --ref-video to maintain subject and setting consistency.
• Audio Integration: Supports standalone audio references via --ref-audio or embedded audio in video references. Audio inputs are capped at 15 seconds total across a maximum of three inputs.

Resolution and Duration

Width and height must be multiples of 32, with a maximum product of $768 \times 1344$ pixels. The model aligns frame requests upward to a specific temporal shape ($5 + 17n$). For example, 22 frames result in approximately 0.917 seconds of video at 24 fps, while 243 frames produce 10.125 seconds.

Technical Implementation and Metal Optimizations

The engine employs several deep-level optimizations to reduce memory footprint and increase throughput on Apple Silicon.

Memory and Weight Management

On M5-class GPUs, transformer weights are mapped directly from safetensor shards to keep the 37 GiB model file-backed and reclaimable. The Qwen text encoder uses a streamed approach with a ring of future layer buffers filled by I/O workers to overlap encoding with Metal execution.

Metal 4 and TensorOps

M5 GPUs utilize native BF16 Metal 4/TensorOps for DiT QKV and attention-output projections. The implementation uses a compact Morton schedule to route Q/K/V directly into head-major attention inputs, avoiding three MPSGraph input transposes.

Int8 Quantization

The default M5 path uses a native int8 MLP engine with dynamic activation quantization and per-output-channel weight scales. This significantly reduces peak tensor storage from 36.4 GiB (BF16) to 25.9 GiB. Further optimizations include:

• Quantized QKV: Reduces denoising time on M5 from 25.80 to 19.32 seconds.
• Fused Kernels: Folds QKV and MLP activation quantization into the preceding gated AdaLN kernel, removing 99 standalone quantizer dispatches per 50-layer forward.

Community Insights and Alternatives

Discussion among users highlights the significant performance gap between native Metal implementations and general-purpose frameworks.

"On my 128GB M4 Max Mac Studio, generating a 15s 480p video with MiniMax H3 in ComfyUI takes an hour and a half."

Users have noted that while high-memory configurations (e.g., 128GB) are ideal for peak performance, the ability to run these models locally on Mac hardware is a significant advancement for local-first workflows, even if specialized CUDA hardware remains faster.