Anthropic Prompt Engineering Guide for Claude's 100k Token Context Window

TL;DR

Anthropic demonstrated that two prompt‑engineering techniques—pulling relevant quotes into a scratchpad and supplying several correctly answered examples—significantly boost Claude's ability to recall specific facts from 70 k–95 k token contexts.

Background: Claude’s 100,000‑Token Context Window

Claude now supports a 100k‑token context window, allowing it to ingest hundreds of pages of technical material or an entire book. As the API scales, users need concrete guidance on how to structure prompts to get the most out of this capacity.

Experimental Design

Goal

Measure how different prompting strategies affect Claude’s accuracy when answering multiple‑choice questions that require recalling a single fact from a very long document.

Data Source

A publicly available U.S. government daily‑issue transcript (July 13, 2023) was chosen because it post‑dates Claude’s training cutoff, minimizing prior knowledge.

Question Generation (Randomized Collage)

Split the document into sections.
Prompt Claude to write five multiple‑choice questions per section, each with three distractors and one correct answer.
Randomly stitch sections together to create long “collage” documents of ~75 k and ~90 k tokens.

Prompting Strategies Tested

Strategy	Description
Base	Simple ask without any examples.
Nongov examples	Two fixed general‑knowledge multiple‑choice examples unrelated to the government text.
Two examples	Two in‑context examples drawn randomly from other sections of the same collage.
Five examples	Same as above but with five examples.

Each strategy was evaluated with and without a <scratchpad> instruction that tells Claude to pull relevant quotes before answering. The answer passage was placed at the beginning, middle, or end of the input to test positional effects.

Key Findings

Baseline Performance

Claude Instant 1.2 answers its own short‑context questions correctly ~90% of the time.
When the answer passage is replaced with a random unrelated section, Claude guesses correctly 34% of the time (above the 25% random baseline).

Impact of Scratchpad and Examples

Adding a scratchpad and examples yields the highest accuracy for both 70 k and 95 k token documents.
Generic, unrelated examples (the “nongov” set) provide no measurable benefit.
More examples improve performance monotonically: five examples outperform two.
The scratchpad incurs a small latency increase but is negligible for Claude Instant, which is already fast.

Positional Effects

Accuracy drops when the relevant passage is at the very end of the prompt and examples are also present, likely because examples increase the distance between the answer and the model’s final reasoning step.
For Claude Instant, performance degrades as the distance between the question and the answer grows; Claude 2 shows a smaller dip, with a slight dip in the middle of 95 k token inputs.

Claude 2 vs. Claude Instant

Claude 2 baseline accuracy is already high (≈0.939). Prompting raises it to 0.961, a 36% reduction in error despite the modest absolute gain.

Practical Prompt‑Engineering Recommendations

Use a scratchpad – Instruct Claude to collect and display relevant quotes before answering; this consistently improves accuracy.
Provide multiple in‑context examples – Prefer five examples drawn from the same long document rather than generic ones.
Place instructions near the end – Keep the scratchpad and example block close to the answer location to minimize distance effects.
Avoid ambiguous references – Ensure questions explicitly name the passage (e.g., “the notice about additional in‑season actions for fisheries”) because “this document” becomes ambiguous in a collage.
Expect diminishing returns at the document tail – If the answer lies at the very end, consider restructuring the prompt so that the answer‑related text appears earlier or the instruction block follows it.

Reproducibility: Anthropic Cookbook

All code, notebooks, and prompt templates used in the study are publicly available in the Anthropic Cookbook:

Full experiment repository – includes data generation, evaluation scripts, and prompt PDFs.
Additional recipes: a Wikipedia search‑and‑retrieval demo and a mock‑PDF upload/summarization guide.

Implications for Developers

The 100k token window unlocks use‑cases such as full‑document QA, book‑length summarization, and large‑scale policy analysis.
Effective prompting is essential; without a scratchpad or relevant examples, accuracy can fall to near‑chance levels for facts buried deep in the context.
The techniques described are lightweight (prompt‑only) and do not require external retrieval systems, making them easy to integrate into existing Claude API workflows.

Limitations and Future Work

The study focuses on Claude Instant 1.2; while Claude 2 shows similar trends, results may differ for future model versions.
Only multiple‑choice factual recall was tested; open‑ended generation or reasoning tasks may require additional strategies.
Positional bias remains a challenge for information at the extreme end of very long prompts; future work could explore dynamic prompt truncation or hierarchical retrieval.

Conclusion

Anthropic’s quantitative case study confirms that extract‑quote scratchpads and multiple in‑context examples are simple yet powerful levers for improving Claude’s factual recall across 70 k–95 k token contexts. By applying these prompt‑engineering patterns, developers can reliably harness Claude’s 100k token window for complex, document‑scale applications.