vLLM AFD Plugin: Disaggregating Attention and FFN for MoE Serving

vLLM has released the vLLM AFD Plugin, an experimental external plugin that implements Attention-FFN Disaggregation (AFD) for Mixture-of-Experts (MoE) models. This architecture separates the Attention and Feed-Forward Network (FFN) components into independently deployed services, enabling them to scale based on their specific resource requirements without altering the vLLM request lifecycle or OpenAI-compatible serving interface.

The Rationale for Attention-FFN Disaggregation

In MoE inference, Attention and FFN paths have fundamentally different operational requirements. Attention is stateful, tied to request scheduling and KV cache management, whereas the FFN (expert) path is dominated by routed computation and all-to-all communication.

AFD addresses four primary system design challenges:

• Independent Scaling: Attention capacity depends on sequence length and KV-cache pressure, while expert capacity depends on token routing and load. AFD allows these paths to use different rank topologies.
• Runtime Responsibilities: By splitting services, the FFN side can operate as a lightweight connector-driven daemon, removing the need for scheduling and KV-cache coordination on the expert side.
• Backend-Specific Communication: A common connector contract allows different hardware backends (e.g., CUDA and Ascend) to implement their own optimized data paths and collective libraries.
• Computation-Communication Overlap: Asynchronous dispatch and MoE ubatching can overlap independent stages, preventing the Attention path from serializing all expert work.

System Architecture

The vLLM AFD Plugin integrates via the vllm.general_plugins entry point and --additional-config channel, requiring no modifications to the vLLM source tree. The runtime consists of three core components:

1. Attention Service

Retains the vLLM scheduler, KV cache, batching, model lifecycle, and sampling path. A plugin-owned model runner installs AFD metadata into the forward context and publishes state (data-parallel, ubatch, layer, and graph) to the FFN service.

2. FFN Service

Operates without request traffic, schedulers, or KV caches. It runs a background loop that receives metadata and activations, executes compute_ffn_output() via a plugin-owned model wrapper, and returns the results to the Attention service.

3. Connector Layer

Acts as the bridge at each split layer, transferring hidden states and execution metadata from Attention to FFN and returning the computed outputs.

Connector and Backend Support Matrix

Connector	Backend	Execution	Recommended Stage	Graph Support
P2pNcclAFDConnector	GPU	Synchronous P2P	Decode	FULL_DECODE_ONLY CUDA graph
CAMP2pAFDConnector	NPU	Synchronous CAMP2P/HCCL	Decode	FULL_DECODE_ONLY ACL graph
CAMAsyncAFDConnector	NPU	Asynchronous CAM	Prefill	Not supported

Performance Benchmarks

Synchronous Decode Throughput (DeepSeek-V3.2 W8A8 on Ascend 910C)

Benchmarks compared a conventional EP64 deployment against AFD deployments using the CAMP2pAFDConnector. Results show that the ratio of Attention to FFN ranks significantly impacts normalized throughput (tokens/s/die).

• 16K Fixed Input: The 64A16F configuration (64 Attention, 16 FFN ranks) achieved 258.9 tokens/s/die, an 11.3% increase over the EP64 baseline (232.6 tokens/s/die). The 48A16F configuration performed worse than the baseline at 220.3 tokens/s/die (-5.3%).
• 32K Fixed Input: The 64A16F configuration achieved 183.3 tokens/s/die, a 9.0% increase over the EP64 baseline (168.2 tokens/s/die). The 48A16F configuration was 10.0% below the baseline.

These results indicate that disaggregation alone does not guarantee gains; the specific allocation of ranks between Attention and FFN is critical.

Asynchronous Prefill Performance (DeepSeek V3.2 W8A8 on Ascend 910C)

An early experiment using CAMAsyncAFDConnector on a 10-layer reduced model compared a DP4PCP8 TP1 baseline with an AFD layout (Attention DP3PCP8 TP1 + FFN EP8).

At a request rate of 12 requests per second, the median Time to First Token (TTFT) decreased from 15.1 seconds to 8.0 seconds, representing a reduction of approximately 47%.

Implementation and Roadmap

Current Support

• Models: Wrappers for DeepSeek V2/V3-family (including DeepSeek V3.2) and GLM MoE DSA.
• Hardware: NVIDIA GPUs and Ascend NPUs.
• Execution Paths: Support for eager, graph, and dual-batch execution; decode-only graph capture for synchronous connectors.
• Requirements: Python 3.10–3.13 and vLLM 0.19.1.

Future Development

• Upstream Alignment: Tracking newer vLLM releases and evaluating model runner v2.
• Execution Flexibility: Expanding graph modes, ubatch counts, and asynchronous stages.
• Production Validation: Publishing stability, accuracy, and latency results on full models and realistic workloads.
• Expanded Coverage: Adding more MoE architectures and backend transports.
• Multimodal Integration: Exploring AFD's application within vLLM-Omni for autoregressive (AR) and Diffusion Transformer (DiT) stages.
• Heterogeneous Hardware: Investigating the deployment of Attention and FFN roles across different accelerator types to reduce TTFT and inter-token latency.