vLLM Speculators: Parallel Drafting for Speculative Decoding

vLLM Speculators: Parallel Drafting for Speculative Decoding

vLLM and the Speculators project have introduced open-source support for three parallel drafting algorithms—P-EAGLE, DFlash, and DSpark—to overcome the latency bottlenecks of traditional autoregressive speculative decoding. By predicting entire blocks of candidate tokens in a single forward pass, these methods decouple drafting latency from the number of speculated tokens, enabling the use of more expressive speculator models and reducing the need for manual parameter tuning.

The Limitations of Autoregressive Drafting

Traditional speculative decoding frameworks, such as EAGLE and MTP, improved token acceptance rates by utilizing the verifier model's internal hidden states. However, advanced iterations like EAGLE-3 still rely on autoregressive drafting, where the speculator must execute a separate forward pass for every single token generated.

This sequential requirement creates two primary production challenges:

  • Model Size Constraints: To prevent the drafting process from consuming the time saved during verification, speculator models must remain extremely small and lightweight.
  • Operational Complexity: Because drafting cost scales linearly with the number of tokens, engineering teams must constantly tune the speculation length (K) based on specific use cases and real-time server loads.

Parallel Drafting: Architectural Shift

Parallel drafting eliminates sequential execution by predicting a candidate block of tokens concurrently in a single forward pass. This shift provides two main advantages:

  1. Increased Expressiveness: Since the speculator only runs once per block, developers can employ larger, deeper architectures that capture more complex context and achieve higher acceptance rates without increasing latency.
  2. Simplified Tuning: Decoupling the cost of drafting from the block length removes the operational burden of hyper-tuning speculation parameters during fluctuating server loads.

While earlier concepts like Medusa and PARD explored parallel drafting, P-EAGLE, DFlash, and DSpark combine this parallel execution with deep verifier-state conditioning.

Technical Comparison of P-EAGLE, DFlash, and DSpark

All three algorithms leverage the verifier model's hidden states to generate parallel drafts, but they differ in their architectural implementation and training strategies.

P-EAGLE

P-EAGLE uses the verifier model's hidden states as input features and maps them across multiple future positions simultaneously to output a sequence of candidate tokens in one step. To manage training costs, it uses draft block sparsification, which drops tokens along the lookahead dimension (K) at a decaying rate to focus optimization on the most critical immediate tokens.

DFlash

DFlash projects verifier hidden states and injects them directly into the speculator model's KV-cache. This conditions the speculator's attention mechanism on the verifier's state without increasing input sequence length, utilizing block diffusion to generate candidate tokens. For training, DFlash uses sequence length sparsification, computing block predictions only at random anchor points along the sequence to preserve GPU memory.

DSpark

DSpark builds on the DFlash backbone with two additional enhancements:

  • Autoregressive Correction Head: A lightweight head that allows future tokens to be more strongly conditioned on past tokens, combining parallel throughput with sequential coherence.
  • Confidence Head: A mechanism that scores draft tokens before they reach the verifier, forwarding only those likely to be accepted to reduce wasted verification compute and improve throughput.

Inference Performance and Benchmarks

Parallel drafting algorithms demonstrate significant performance gains over EAGLE-3 across various models and tasks:

Model Algorithm Use Case Hardware
Qwen3-8B P-EAGLE Math reasoning (GSM8k) 1xA100
Qwen3-30B-A3B DFlash Coding (HumanEval) 2xA100
gemma-4-31B-it DSpark Coding (HumanEval) 2xA100

Production Integration with vLLM

Parallel drafting is integrated into vLLM via the Speculators repository. This ecosystem provides a unified framework for training and evaluating these models. Users can launch a parallel-backed speculative engine by specifying the model and method in the speculative-config flag during vLLM initialization:

vllm serve Qwen/Qwen3-30B-A3B \
  --tensor-parallel-size 2 \
  --reasoning-parser qwen3 \
  --speculative-config '{
    "model": "RedHatAI/Qwen3-30B-A3B-speculator.dflash",
    "num_speculative_tokens": 7,
    "method": "dflash"
  }'

Because speculative decoding uses rejection sampling to preserve the verifier model's output distribution, the final output quality remains mathematically identical to standard decoding.

Sources