Anthropic Effective Context Engineering for AI Agents

Anthropic has introduced the concept of context engineering, a shift from traditional prompt engineering toward the strategic curation and maintenance of the total set of tokens provided to a large language model (LLM) during inference. This approach is critical for building capable AI agents that operate over multiple turns and long time horizons, where managing the holistic state—including system instructions, tools, and message history—is more important than simply refining the wording of a prompt.

The Necessity of Context Engineering

Context engineering is required because LLMs possess a finite "attention budget" and are subject to context rot, a phenomenon where a model's ability to accurately recall information decreases as the number of tokens in the context window increases.

This degradation stems from the transformer architecture's $n^2$ pairwise relationships between tokens, which stretches attention thin as context grows. Additionally, because training data typically consists of shorter sequences, models have fewer specialized parameters for context-wide dependencies. Consequently, context must be treated as a finite resource with diminishing marginal returns, where the goal is to find the smallest possible set of high-signal tokens to maximize the likelihood of a desired outcome.

Anatomy of Effective Context

To maximize the utility of the attention budget, Anthropic recommends optimizing three primary components of the context:

System Prompts

System prompts should aim for the "right altitude"—a balance between brittle, hardcoded logic and overly vague guidance. They should be specific enough to guide behavior effectively but flexible enough to provide strong heuristics. Anthropic suggests organizing prompts into distinct sections (e.g., <background_information>, <instructions>) using XML tagging or Markdown headers to delineate information.

Tools

Tools should be self-contained, robust, and token-efficient. A common failure mode is the use of "bloated tool sets" that create ambiguity for the agent. Curating a minimal viable set of tools ensures more reliable maintenance and pruning of context over long interactions.

Examples

Rather than providing an exhaustive list of edge cases, developers should curate a set of diverse, canonical examples (few-shot prompting) that portray the expected behavior of the agent.

Context Retrieval and Agentic Search

Anthropic observes a shift from embedding-based pre-inference retrieval toward "just-in-time" context strategies.

• Just-in-Time Retrieval: Agents maintain lightweight identifiers (such as file paths or web links) and use tools to dynamically load data into context at runtime. For example, Claude Code uses this approach to analyze large databases by writing targeted queries and using Bash commands like head and tail to avoid loading full data objects.
• Progressive Disclosure: This allows agents to incrementally discover relevant context through exploration, using metadata (like folder hierarchies and timestamps) to inform subsequent decisions.
• Hybrid Strategy: Some agents combine pre-computed data retrieval for speed with autonomous exploration for depth. Claude Code utilizes this by loading CLAUDE.md files up front while using glob and grep for just-in-time navigation.

Managing Long-Horizon Tasks

For tasks spanning hours or requiring token counts that exceed the context window, Anthropic outlines three primary techniques to prevent context pollution:

Compaction

Compaction involves summarizing a conversation as it nears the context limit and reinitiating a new window with that summary. In Claude Code, the model preserves architectural decisions and unresolved bugs while discarding redundant tool outputs. A "light touch" version of this is tool result clearing, which removes raw results of old tool calls from the history.

Structured Note-Taking

Also known as agentic memory, this involves the agent writing notes to a persistent external memory (e.g., a NOTES.md file) and pulling them back into context as needed. This allows agents to maintain state across resets, as seen in Claude's ability to track objectives and combat strategies while playing Pokémon.

Sub-Agent Architectures

This architecture uses a lead agent to coordinate a high-level plan while specialized sub-agents handle focused tasks with clean context windows. Sub-agents perform deep work and return only a condensed summary (typically 1,000–2,000 tokens) to the lead agent, ensuring a clear separation of concerns.

Strategy	Best Use Case
Compaction	Tasks requiring extensive back-and-forth conversational flow
Note-Taking	Iterative development with clear milestones
Multi-Agent	Complex research and analysis requiring parallel exploration