z-lab/dflash

DFlash: Block Diffusion for Flash Speculative Decoding

What it solves

DFlash addresses the slow speed of token-by-token generation in large language models (LLMs) by implementing a more efficient method of speculative decoding. It enables high-quality parallel drafting of tokens, reducing the time it takes for a target model to generate a response.

How it works

It uses a lightweight block diffusion model to act as a draft model. This draft model predicts multiple tokens in parallel rather than one at a time, which the target LLM then verifies in a single pass. The system supports multiple backends including Transformers, MLX (for Apple Silicon), and OpenAI-compatible servers like SGLang and vLLM.

Who it’s for

Developers and researchers working with LLM inference who want to increase generation throughput and reduce latency through speculative decoding.

Highlights

  • Parallel Drafting: Uses block diffusion to generate multiple token candidates simultaneously.
  • Broad Model Support: Compatible with a wide range of models including Qwen, Gemma, MiniMax, Kimi, and Llama-3.1.
  • Flexible Deployment: Works across different hardware and software stacks, including local MLX for Mac and server-side vLLM/SGLang.
  • Integrated Benchmarking: Includes built-in tools to evaluate performance on datasets like GSM8K and HumanEval.

Related

  • Project
    bstnxbt/dflash-mlx

    A speculative decoding implementation for Apple Silicon that uses block-diffusion draft models to accelerate LLM inference without losing output quality.

  • Dispatch
    vLLM Speculators: Parallel Drafting for Speculative Decoding

    vLLM and the Speculators project introduce open-source support for P-EAGLE, DFlash, and DSpark, moving beyond autoregressive drafting to generate candidate token blocks in parallel for faster LLM inference.

  • Project
    ARahim3/mlx-dspark

    A high-performance inference engine for Apple Silicon that uses DSpark and DFlash speculative decoding to accelerate LLM generation speed losslessly.

  • Dispatch
    vLLM speculative decoding on AMD GPUs: performance and methods

    vLLM adds speculative decoding to AMD Instinct MI300X/MI355X GPUs, letting a lightweight draft model propose multiple tokens that the target model verifies in a single pass, which can double or more the output-token throughput depending on the draft method, model family, and proposal length.

  • Project
    inclusionAI/LLaDA2.X

    A series of discrete diffusion Large Language Models scaled up to 100B parameters, offering high-speed parallel decoding for text generation.