ARahim3/mlx-dspark

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

What is mlx‑dspark?

mlx‑dspark is a Python package (and a native macOS app) that lets you run speculative decoding – a technique that speeds‑up text generation – on Apple‑silicon Macs. It ships two “draft‑generation” engines that were originally released by DeepSeek (DSpark) and z‑lab (DFlash). Both engines are loss‑less: they propose several tokens ahead, then the real model verifies each token, so the final output is identical to ordinary decoding, only faster.

How it works

  • Speculative decoding: a fast “drafter” predicts a short sequence of tokens. The full model then checks those tokens; if they all match, the whole block is accepted, otherwise the model falls back to normal decoding. Because the drafter is cheap, the overall throughput can be 2‑5× higher.
  • Two drafter families:
    • DSpark – semi‑autoregressive drafter from DeepSeek’s DeepSpec codebase.
    • DFlash – block‑diffusion drafter from z‑lab.
  • The package automatically picks the best drafter for the model you load, or you can force a specific one.
  • It measures your Mac’s performance on first use and chooses an optimal draft‑size (--max‑draft auto).

What you can do with it

  1. CLI generation – mlx-dspark generate --model <repo> --prompt "…".
  2. Local OpenAI‑compatible server – mlx-dspark serve … gives an HTTP API that works with any client that talks the OpenAI (or Anthropic) API (Ollama, Open WebUI, SillyTavern, etc.).
  3. macOS GUI app – a click‑through interface for chat, model management, live telemetry, and one‑click setup of coding‑assistant agents.
  4. Integration with Claude Code and Codex – the server can act as a backend for Anthropic’s Claude Code or OpenAI’s Codex APIs.

Supported models

The README lists a table of models that have a pre‑registered drafter and measured speed‑ups, e.g.:

  • Gemma‑4 12B (8‑bit) – up to 4.24× faster.
  • Qwen‑3.8‑27B (8‑bit, DFlash 2) – up to 4.22×.
  • MiniCPM5‑2B – up to 4.21×.
  • LFM2.5‑1.2B – up to 3.83×.
  • Many others (Qwen, Ornith, Nemotron, Bonsai, etc.). Even models not in the table run with a “drafter‑free” lookup mode (--mode auto).

Installation & requirements

  • Python ≥ 3.10 on Apple Silicon.
  • pip install mlx-dspark (installs the MLX runtime ≥ 0.32 automatically).
  • Optionally install the GUI via Homebrew Cask or download the DMG from the releases page.
  • No extra server framework – the HTTP API is built on the Python standard library.

Key command‑line flags (most common)

Flag Purpose
--model <repo> HF repo or local path of the target model.
`--mode auto dspark
--max-draft auto Let the tool auto‑tune the draft length for your Mac.
--max-batch N Batch up to N concurrent requests (≈2.5× aggregate throughput).
--kv-bits 8 Quantise the KV cache to save memory for long contexts.
--api-key <key> Simple auth if you expose the server beyond localhost.
--trust‑remote‑code Allow loading checkpoints that ship custom Python code.

Why it matters

Speculative decoding is a cutting‑edge technique that lets consumer‑grade LLMs run noticeably faster on a single Mac without sacrificing answer quality. By exposing a standard OpenAI/Anthropic API, mlx‑dspark makes it easy to plug these faster local models into existing tooling and agents, keeping everything on‑device and private.

All details above are taken directly from the repository’s README; no external information has been added.

Related

  • Project
    bstnxbt/dflash-mlx

    A speculative decoding implementation for Apple Silicon that uses block-diffusion draft models to accelerate LLM inference without losing output quality.

  • Project
    incoai/splash

    A high-performance local inference engine for Apple silicon that uses speculative decoding and Metal kernels to run LLMs and coding agents faster than traditional engines.

  • Project
    youssofal/MTPLX

    A native Mac application and CLI that accelerates local LLM inference on Apple Silicon using Multi-Token Prediction (MTP) for faster decoding speeds.

  • Project
    raullenchai/Rapid-MLX

    A high-performance LLM inference server for Apple Silicon that optimizes MLX for faster local inference and reliable tool-calling for coding agents.

  • Dispatch
    DeepSeek DSpark: Accelerating LLM Inference by 60-85%

    DeepSeek has open-sourced DSpark, an inference optimization technique that achieves 60-85% faster generation speeds by improving upon speculative decoding.