DeepSeek-V4 Release Notes: Efficient 1M-Token Context for AI Agents

DeepSeek has released DeepSeek-V4, a series of Mixture-of-Experts (MoE) models designed to solve the efficiency bottlenecks of long-context agentic workloads. While general benchmarks are competitive, the primary innovation lies in a new attention architecture that drastically reduces KV cache memory and inference FLOPs, making 1M-token contexts computationally viable for agents.

Efficient Long-Context Inference via Hybrid Attention

DeepSeek-V4 achieves a 1M-token context window by splitting attention into two interleaved mechanisms: Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA). This architecture reduces the KV cache size to approximately 2% of that required by standard grouped query attention (GQA) with 8 heads in bfloat16.

Compressed Sparse Attention (CSA)

CSA compresses KV entries by 4x along the sequence dimension using softmax-gated pooling with a learned positional bias. To maintain performance, a "lightning indexer" (using FP4, ReLU-scored multi-head dot products) selects the top-k compressed blocks per query. A sliding-window branch is used to handle the most recent uncompressed tokens.

Heavily Compressed Attention (HCA)

HCA employs a more aggressive 128x compression of KV entries. Because the resulting compressed sequence is so short, the model can use dense attention over every compressed block without the need for sparse selection. Like CSA, HCA includes a sliding-window branch for recency.

Architectural Implementation

In the DeepSeek-V4-Pro model (61 layers), layers 0–1 use HCA, while layers 2–60 alternate between CSA and HCA. The final MTP block uses sliding-window attention only. To further reduce memory, the model uses FP8 storage for most KV entries and FP4 for the CSA lightning indexer.

Agent-Specific Optimizations and Infrastructure

Beyond the attention mechanism, DeepSeek-V4 incorporates three specific design choices to improve the reliability and coherence of AI agents during multi-step tool use.

Interleaved Thinking Across Tool Calls

Unlike previous versions that flushed reasoning traces upon receiving a new user message, V4 preserves reasoning content across user message boundaries when tool calls are involved. This allows the model to maintain a cumulative chain of thought over long-horizon tasks, though the standard behavior of flushing reasoning is maintained for non-tool conversational use to save context.

Dedicated Tool-Call Schema

To reduce parsing errors and escaping failures common with JSON-in-string formats, V4 introduces:

• A special |DSML| token.
• An XML-based tool-call format.
• A parameter system that explicitly separates string parameters (string="true") from structured JSON parameters (string="false").

DSec Sandbox for RL Training

Agent behaviors were trained using Reinforcement Learning (RL) within DeepSeek Elastic Compute (DSec), a Rust-based platform. DSec supports function calls, containers, microVMs (Firecracker), and full VMs (QEMU) via a single Python SDK. Key features include layered 3FS storage for fast image loading and preemption-safe trajectory replay, ensuring RL rollouts are not delayed by container startup or interrupted training steps.

Performance and Benchmarks

DeepSeek-V4-Pro-Max demonstrates parity with frontier closed models on agent-centric benchmarks:

• SWE Verified: 80.6 resolved (comparable to Opus-4.6-Max at 80.8 and Gemini-3.1-Pro at 80.6).
• MCPAtlas Public: 73.6 (second only to Opus-4.6-Max at 73.8).
• Toolathlon: 51.8 (outperforming Gemini-3.1-Pro at 48.8 and K2.6 at 50.0).
• Terminal Bench 2.0: 67.9 (ahead of GLM-5.1 and K2.6, though behind GPT-5.4-xHigh).

In an internal R&D coding benchmark involving PyTorch, CUDA, Rust, and C++, V4-Pro-Max achieved a 67% pass rate. Long-context retrieval (MRCR 8-needle) remains above 0.82 through 256K tokens and holds at 0.59 at 1M tokens.

Model Availability and Usage

DeepSeek has released four checkpoints on the Hugging Face Hub:

Model	Total Params	Active Params	Type
DeepSeek-V4-Pro	1.6T	49B	Instruct
DeepSeek-V4-Flash	284B	13B	Instruct
DeepSeek-V4-Pro-Base	1.6T	49B	Base
DeepSeek-V4-Flash-Base	284B	13B	Base

Instruct models support three reasoning modes: Non-think (fast), Think High (explicit reasoning in <think> blocks), and Think Max (maximum effort, requiring at least 384K context window). Recommended sampling parameters are temperature=1.0 and top_p=1.0.