Optimizing MiniMax M3 on AMD Instinct MI355X

vLLM has significantly optimized the serving performance of MiniMax M3 on AMD Instinct MI355X, achieving up to a 3.14× increase in output throughput for MXFP8 standard serving at low concurrency. This was accomplished by iteratively identifying and resolving bottlenecks across the entire stack, from GEMM tiling and kernel fusion to distributed system orchestration.

Performance Benchmarks on AMD Instinct MI355X

Optimization efforts have led to substantial improvements in throughput and latency for MiniMax M3 across various configurations:

  • MXFP8 Standard Serving (Concurrency 32): Output tokens/s/GPU rose from 109.1 to 342.4 (3.14×). Median Time to First Token (TTFT) decreased from 1.46 to 0.67 seconds, and mean Time Per Output Token (TPOT) fell from 69.1 to 22.1 milliseconds.
  • MXFP8 Standard Serving (Concurrency 128): Output tokens/s/GPU rose from 297.8 to 623.7 (2.09×). Median TTFT fell from 3.53 to 1.54 seconds, and mean TPOT fell from 100.7 to 48.8 milliseconds.
  • MXFP4 Serving (Concurrency 128): Throughput rose from 212.1 to 716.8 output tokens/s/GPU on TP4/EP1. A further optimization to TP2/EP1 reached 943.5 output tokens/s/GPU, representing a 4.45× gain over initial results.
  • Speculative Decoding (EAGLE3): Reached 682.4 output tokens/s/GPU at concurrency 128 on TP4/EP1.
  • P/D Disaggregation: Reached 6,370.5 total tokens/s/GPU at concurrency 512 with a 1.32 seconds median TTFT.

Kernel and Operator Optimizations

Performance gains were driven by a systematic approach to local shape analysis and the elimination of redundant computations.

Local Shape and Tiling

vLLM optimized the GEMM launcher to handle different regimes for prefill (large-M) and decode (small-M). By selecting narrower N tiles for decode and larger K steps to reduce loop iterations, vLLM improved TP8 8K/1K output throughput by 7.8–9.4%. Additionally, reordering grouped-MoE programs allowed neighboring programs to reuse activation rows and expert-weight tiles from the GPU cache, yielding 1.08×–1.46× gains on TP4 tests.

Shared Expert Fusion

MiniMax M3's shared expert was originally executed as a separate dense MLP path. vLLM fused the shared expert into the routed expert table, allowing grouped GEMMs to handle both simultaneously. This removed separate launches and intermediate traffic, improving output throughput by 30.2% at concurrency 1 and 5.6% at concurrency 128.

Sparse Attention and Index Reuse

To reduce the overhead of sparse attention, vLLM implemented index sharing, where later layers reuse the top-k block decisions of previous layers. This reduced mean TPOT by approximately 10% at concurrency 1. Furthermore, vLLM developed a sparse-page adapter to map MiniMax M3's 128-token logical blocks to AITER's 16-token pages without copying KV data, improving output throughput by 5.56% (MXFP8) to 6.93% (MXFP4) at TP4, concurrency 256.

Distributed System and Speculative Execution

Optimizations extended beyond individual kernels to the coordination of multiple GPUs and the integration of speculative decoding.

EAGLE3 Speculative Decoding

vLLM integrated the EAGLE3 draft model on AMD hardware, implementing request-level index batching to process all draft positions together. This improved the index kernel by up to 48.9%. Additionally, extending AITER sparse paged attention to multi-token verification improved output throughput by 7.90% (MXFP8) and 8.32% (MXFP4) in TP4 tests.

Prefill/Decode (P/D) Disaggregation

vLLM implemented P/D disaggregation by deriving precise transfer geometry and byte offsets per layer to ensure correct KV handoffs. System tuning revealed that moving all workers to TP4 and using a ratio of two TP4 prefill workers to one TP4 decode worker for 8K/1K workloads achieved 6,370.5 total tokens/s/GPU at concurrency 512 with a 1.32 seconds median TTFT.

Agentic Workload Analysis

Using the AgentX benchmark for agentic coding (which involves irregular outputs and reusable prefixes), vLLM tested a configuration with MXFP4, EAGLE3-GQA, and prefix caching on TP4 at concurrency 28. The system delivered 127.4 output tok/s/GPU and 509.5 total output tok/s, with a 645 ms p50 TTFT and 41.3 ms p50 TPOT. Analysis showed a 92.1% realized GPU cache hit rate and 88.5% KV-cache utilization, suggesting that future optimizations should focus on prefix alignment and eviction policies rather than further GEMM tuning.

Optimization Methodology Checklist

To optimize future model serving paths, vLLM proposes the following technical checklist:

  1. Local Shape Analysis: Record local shape histograms after sharding, including replicated heads and routed-token counts.
  2. Repetition Estimation: Calculate per-layer work multiplied by layers, output tokens, and active requests.
  3. Byte Plane Separation: Distinguish between compute tensors, persistent state, and communication.
  4. Fast Path Verification: Record eligibility conditions for every fast path and verify actual execution via dispatch traces.
  5. Correctness Gating: Implement correctness checks alongside performance gates.
  6. Iterative Profiling: Once leaf kernels flatten, inspect higher-level queues, ownership, cache capacity, and OS limits.

Sources