XiaomiMiMo/MiMo-V2-Flash

What it solves

MiMo-V2-Flash addresses the high computational costs and memory requirements of running large-scale language models. It provides a high-performance, efficient solution for long-context reasoning and complex agentic tasks, such as coding and web development, while significantly reducing inference costs.

How it works

The model uses a Mixture-of-Experts (MoE) architecture with 309B total parameters, but only 15B active parameters per token. It achieves efficiency through:

• Hybrid Attention: Interleaves sliding window attention with global attention to reduce memory usage by nearly 6x.
• Multi-Token Prediction (MTP): A built-in module that predicts multiple future tokens simultaneously, tripling output speed.
• Advanced Training: Utilizes Multi-Teacher On-Policy Distillation (MOPD) and large-scale reinforcement learning on code-based tasks to improve reasoning and agentic capabilities.

Who it’s for

This model is designed for developers and researchers building high-performance AI agents, long-context applications, or systems requiring fast, cost-effective inference on complex reasoning tasks.

Highlights

• 256k context window for processing massive documents or codebases.
• Optimized for agentic workflows, including tool use and multi-turn reasoning.
• Native support for FP8 mixed precision inference.
• Includes open-source MTP weights to support community research.
• Proven performance on benchmarks like SWE-Bench and complex math/coding tasks.