PipeNetwork/kimi-k3-mlx

MLX port of moonshotai/Kimi-K3 (2.78T multimodal MoE): streaming converter, REAP expert pruning, and per-language expert-overlap analysis

What it solves

This project provides an MLX port of Moonshot's Kimi-K3, a massive native-multimodal Mixture-of-Experts (MoE) model. Because the full model is too large to fit on any existing Apple Silicon machine (requiring ~883 GB at its smallest unpruned tier), the project implements a streaming converter and a pruning system to make the model runnable on high-end Mac hardware.

How it works

  • Architecture Porting: It implements Kimi-K3's specific components in MLX, including SiTU-GLU activations, Attention Residuals (AttnRes), LatentMoE, and a specialized vision tower (MoonViT) for 3D/video capabilities.
  • Streaming Conversion: A custom converter processes the 5.6 TB (bf16) model in chunks to avoid loading the entire model into RAM.
  • REAP Pruning: It uses the REAP method to score expert saliency via a calibration set, allowing the model to be pruned down to a size that fits within the 512 GB memory ceiling of the M3 Ultra.
  • Quantization: It supports native mxfp4 quantization, which allows the source weights to be reinterpreted into MLX with zero arithmetic error.
  • Multimodal Integration: A custom wrapper handles the expansion of image placeholders into full feature blocks, ensuring the vision tower's output is correctly merged into the text tower's input.

Who it’s for

Developers and researchers with high-end Apple Silicon hardware (specifically those with large unified memory capacities) who want to run Moonshot's Kimi-K3 multimodal model locally.

Highlights

  • Bit-Exact Port: The mxfp4 tier is bit-exact compared to the source weights.
  • LMM Capabilities: Supports image and video input via a 27-layer MoonViT vision tower.
  • Memory Optimization: Implements expert pruning to fit a 2.78T parameter model onto consumer-grade (albeit high-end) Mac hardware.
  • Verification Suite: Includes comprehensive tests for vision parity, placeholder expansion, and conversion round-trips.

Related

  • Dispatch
    Moonshot AI Kimi-K3 Release

    Moonshot AI has released Kimi-K3, the first open-weights 3T-class model designed for frontier intelligence in coding, reasoning, and long-horizon knowledge work.

  • Project
    MoonshotAI/Kimi-K3

    Kimi K3 is an open-weight, 2.8T-parameter native multimodal MoE model designed for long-horizon coding, deep research, and complex agentic knowledge work.

  • Dispatch
    vLLM Kimi K3 Support Preview

    vLLM is preparing day-0 open-source serving support for Moonshot AI's Kimi K3, a 2.8-trillion-parameter model featuring a hybrid KDA/full-attention architecture and native vision support.

  • Project
    leonickson1/Swiftlet

    A Swift and Metal runtime that enables large MoE models like Qwen to run on Apple devices by streaming experts from storage to RAM on demand.

  • Dispatch
    vLLM Kimi K3 Support

    vLLM has released day-0 support for Kimi K3, a 2.8-trillion-parameter multimodal MoE model, featuring optimizations for Kimi Delta Attention and DSpark speculative decoding to achieve up to 370 tok/s.