MiniMax M3 vLLM Support: Day-0 Serving for 1M-Token Multimodal Reasoning

MiniMax M3 vLLM Support: Day-0 Serving for 1M-Token Multimodal Reasoning

vLLM has announced day-0 support for the MiniMax M3 model family, including BF16 and MXFP8 checkpoints. This release enables the serving of workloads requiring million-token contexts, native multimodal reasoning, and agentic workflows with tool use and controllable thinking behavior.

MiniMax Sparse Attention (MSA) and 1M-Token Context

MiniMax Sparse Attention (MSA) is the core architectural innovation that makes 1M-token context practical for serving. Instead of dense attention over the entire KV cache, MSA uses an index path to score 128-token KV blocks and select only the most relevant blocks for the actual attention computation.

MSA Execution Process

Every query token follows a three-step process to minimize attention work:

  1. Block Scoring: A small index head scores candidate KV blocks.
  2. Block Selection: The system selects top-k blocks based on learned scores and configured rules (e.g., the current recipe uses local_blocks=1 to ensure the local-window block near the query token is preserved).
  3. Sparse GQA: Online-softmax attention is run only over the selected KV blocks.

KV Cache Integration

MiniMax M3 stores KV data as ordinary paged KV, allowing vLLM to maintain a simple cache manager while applying sparsity during the computation path. This architecture supports prefix caching and chunked prefill, which are critical for workloads that reuse long prompts such as codebases or multi-turn agent traces.

Multimodal Capabilities and Tool Use

MiniMax M3 is a natively multimodal model capable of processing image and video inputs alongside text.

Multimodal Request Path

To optimize GPU utilization, vLLM implements a path where CPU-side preprocessing (decoding frames, sampling video, resizing, and normalizing images) happens upstream. This ensures that the vLLM worker receives ready-to-run tensors, reserving GPU time for inference rather than media processing.

Agentic Workflows

The release includes model-specific parsers and controls to support agentic behavior:

  • Tool and Reasoning Parsers: The minimax_m3 tool-call and reasoning parsers convert model-specific text conventions into structured API responses.
  • Thinking Mode: Users can control thinking behavior via chat_template_kwargs using modes such as enabled, disabled, or adaptive.

Performance Optimizations and Serving Stack

Speculative Decoding with EAGLE3

Day-0 support includes EAGLE3 speculative decoding using the Inferact/MiniMax-M3-EAGLE3 draft model. To maintain low latency, vLLM updated MSA decode kernels to support a uniform decode_query_len, allowing speculative verification to use the decode-specialized split-K path instead of falling back to slower prefill kernels.

Hardware-Specific Backends

Serving is optimized for both NVIDIA and AMD hardware:

  • NVIDIA: Uses the default attention backend for MSA and the FlashInfer backend for the vision encoder. MXFP8 checkpoints utilize the DeepGEMM MXFP8 MoE backend on Blackwell-class systems and Marlin MXFP8 on Hopper-class systems.
  • AMD ROCm: Uses the Triton attention backend for MSA and the ROCM_AITER_FA backend for the vision encoder, verified on MI300 and MI350 Series GPUs.

Kernel Fusions

To reduce HBM round trips and launch overhead, vLLM implemented several fusions, including QKNorm + RoPE + KV insert and GemmaNorm combined with AllReduce.

RL Post-Training Integration

Beyond inference, the vLLM serving path enables reinforcement learning post-training. NVIDIA NeMo RL now utilizes vLLM as a non-colocated generation backend for MiniMax M3, supporting Group Relative Policy Optimization (GRPO) post-training on BF16 checkpoints using expert parallelism.

Technical Roadmap

Future optimizations for MiniMax M3 in vLLM include:

  • FP8 Indexer and KV-Cache: Reducing memory pressure and increasing batch capacity.
  • TRTLLM-Gen MoE: Enhancing Blackwell performance for MXFP8 expert execution.
  • Context Parallelism: Scaling prefill for contexts that exceed the capacity of a single node.
  • Disaggregated Serving: Expanding NIXL and prefill/decode disaggregation recipes.

Sources