ARahim3/mlx-dspark

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

What it solves

mlx-dspark is a high-performance inference engine for Apple Silicon that implements speculative decoding to accelerate the generation speed of Large Language Models (LLMs). It eliminates the slow token-by-token generation process by using "drafters" (smaller, faster models) to predict multiple tokens at once, which the main target model then verifies in a single pass. This process is lossless, meaning the final output is identical to therunning the target model alone, but significantly faster.

How it works

The project implements two EAGLE-family speculative-decoding drafters natively on MLX:

  • DSpark: A semi-autoregressive drafter from DeepSeek's DeepSpec codebase.
  • DFlash: A block diffusion drafter from z-lab.

It also supports n-gram lookup speculation (drafter-free) for any model. The engine automatically calibrates to the specific Mac hardware (M1 through M4) to determine the optimal draft length for maximum speedup. It features continuous batching for concurrent requests and prefix caching to avoid re-processing long system prompts in multi-turn conversations.

Who it’s for

  • Mac users running local LLMs who want significantly higher tokens-per-second without sacrificing model quality.
  • Developers building AI agents (like Claude Code or pi) that require fast prefill and generation on local hardware.
  • Researchers benchmarking speculative decoding methods on Apple Silicon.

Highlights

  • Lossless Acceleration: Guaranteed identical output to the target model.
  • Broad Model Support: Optimized for Gemma-4, Qwen3, Muse-Glimmer, Ornith-1.0, and Nemotron.
  • Dual API Support: Serves both OpenAI-compatible and Anthropic Messages API endpoints on a single port.
  • Native Mac App: Includes a GUI for model management, live telemetry, and a "Race" mode to compare decoding speeds.
  • Agent Integration: Specifically optimized to drive tools like Claude Code and pi locally.
  • C-level Performance: Achieves up to 3.37x speedup on specific models (e.g., Qwen3.8-27B) on M4 Pro hardware.

Related

  • Project
    bstnxbt/dflash-mlx

    A speculative decoding implementation for Apple Silicon that uses block-diffusion draft models to accelerate LLM inference without losing output quality.

  • Dispatch
    DeepSeek DSpark: Accelerating LLM Inference by 60-85%

    DeepSeek has open-sourced DSpark, an inference optimization technique that achieves 60-85% faster generation speeds by improving upon speculative decoding.

  • Project
    z-lab/dflash

    A lightweight block diffusion model for speculative decoding that enables efficient, high-quality parallel drafting to speed up LLM generation.

  • Project
    youssofal/MTPLX

    A native Apple Silicon runtime that accelerates local LLMs by utilizing built-in multi-token prediction heads to increase decoding speed without changing output quality.

  • Project
    antirez/ds4

    A native inference engine optimized for DeepSeek V4 and GLM 5.2, featuring SSD streaming and speculative decoding to run large models on consumer hardware.