antirez/h3.c

What it solves

It provides a native Metal-based inference engine for the MiniMax-H3 model on Apple Silicon, enabling high-performance generation of video and audio from text prompts, images, or other media references on macOS devices.

How it works

The project implements the MiniMax-H3 architecture using Apple's Metal framework for GPU acceleration. It supports various optimization techniques to balance speed and quality, including:

• Denoising Control: Adjustable denoising passes and velocity extrapolation (reuse) to reduce compute time.
• Layer Thinning: The ability to run a subset of transformer blocks to lower memory and compute requirements.
• SSD Streaming: Loading model blocks from SSD to GPU memory on-demand to support low-memory devices.
• Reference Conditioning: Support for first/last-frame anchors (FL2VA) and ordered media references (Ref2VA) for image-to-video or video-to-video tasks.
• Memory Optimizations: Fused gates and AdaLN operations to minimize memory dispatch and global rereads.

Who it’s for

Developers and creators using Apple Silicon (M-series chips) who want to run the MiniMax-H3 generative model locally for high-quality video and audio synthesis.

Highlights

• Native Apple Silicon Support: Optimized specifically for M3 Max and M5 Max chips.
• Flexible Generation: Supports prompt-to-video/audio, image-to-video, and video-to-video with audio preservation or replacement.
• H3-Specific Optimizations: Includes token reduction, internal canvas scaling, and specialized RoPE adaptation for low-resolution previews.
• Interactive Session: An Iris-style interactive CLI that keeps model components in memory for faster iterative prompting.
• Low-Memory Mode: SSD streaming allows the model to run on devices with limited unified memory.