youssofal/MTPLX

3x faster speeds on MLX | Qwen 3.8 27B | Native MTP Speculative Decoding On Apple Silicon With No External Drafter.

What it solves

MTPLX accelerates the execution of local large language models (LLMs) on Apple Silicon Macs. It specifically addresses the inefficiency of standard autoregressive decoding by utilizing multi-token prediction (MTP) heads already present in modern models (like Qwen 3.5/3.6/3.8), allowing the model to draft and verify multiple tokens at once without needing a separate, memory-consuming draft model.

How it works

MTPLX uses the model's own built-in MTP heads to draft several tokens ahead of the current position. It then verifies these drafted blocks in a single batched forward pass. To ensure the output remains mathematically identical to standard sampling, it employs exact rejection sampling with residual correction based on the Leviathan and Chen theorem. This allows it to increase speed (reported up to 2.24x) while maintaining the same output distribution regardless of temperature or top_p settings.

Who it’s for

  • Mac users with Apple Silicon (M1 or newer) and macOS 14+ who want to run local LLMs faster.
  • Developers who need an OpenAI or Anthropic-compatible API server for local agents, coding assistants (like Claude Code or Cline), or RAG pipelines.
  • Model creators who want to train, verify, and publish MTP-enabled models using the integrated "Forge" tool.

Highlights

  • Native Mac Experience: Available as a DMG app with automated hardware checks, model recommendations, and fan control.
  • LTP-Native Speed: Leverages built-in MTP heads for faster decoding without additional RAM overhead.
  • Auto-tuning: Automatically measures the optimal draft depth for a specific Mac's hardware (chip, bandwidth, thermals) to maximize tokens per second.
  • Unified Server: Provides an OpenAI/Anthropic-compatible API that can serve chat models, embedding models, and rerankers from a single daemon.
  • Forge Tooling: Includes a utility to convert Hugging Face repos to MLX, train MTP adapters, and verify speed gains.
  • Session Management: Features a warm-prefix session bank and SSD session cache for near-instant restoration of multi-turn chats.

Related

  • Project
    raullenchai/Rapid-MLX

    A high-performance local AI engine for Apple Silicon that provides OpenAI and Anthropic-compatible APIs for running LLMs, audio, and video models natively on M-series Macs.

  • Project
    jjang-ai/vmlx

    A self-hosted inference server for Apple Silicon that provides OpenAI and Anthropic compatible APIs for running LLMs, VLMs, and image generation models locally.

  • Project
    lmstudio-ai/mlx-engine

    An Apple MLX-based LLM engine for LM Studio that enables optimized local inference of text and vision models on macOS.

  • Project
    bstnxbt/dflash-mlx

    A speculative decoding implementation for Apple Silicon that uses block-diffusion draft models to accelerate LLM inference without losing output quality.

  • Project
    jjang-ai/mlxstudio

    A native macOS app for Apple Silicon that provides a user-friendly interface to run LLMs, vision models, and image generation locally using the MLX framework.