HANDBOOK.md: Benchmark Reveals Long Policy Documents Fail to Govern AI Agents

Long Policy Documents Do Not Reliably Govern AI Agents

Research presented in the HANDBOOK.md benchmark reveals that language-model agents cannot be trusted to let long, standing instructions—such as system prompts, policy files, or skills documents—consistently govern their actions over an extended tool-use horizon. Despite the ability of frontier models to process large context windows, the benchmark shows a significant gap between the capacity to hold tokens and the ability to adhere to complex, binding rules.

Under strict grading, where a trial passes only if every single programmatic criterion is satisfied, the best of thirty evaluated model configurations passed only 36.2% of trials, while most frontier configurations remained below 25%. This suggests that simply placing a policy document in the context window is an insufficient mechanism for ensuring agent compliance in enterprise environments.

The HANDBOOK.md Benchmark Methodology

HANDBOOK.md is designed to simulate how enterprise employees follow company handbooks. The benchmark consists of 65 agentic tasks across five domains: finance, medical billing, insurance, logistics, and HR, spanning ten fictional companies.

Environment and Constraints

  • Tooling: Agents operate in a self-contained company environment using the Model Context Protocol (MCP), with access to mock email, chat, calendar, issue-tracking, and commerce services.
  • Policy Length: Each task is governed by an expert-written standard operating procedure (SOP) ranging from 20 to 124 pages.
  • Anti-Memorization: To prevent models from relying on training data, every task modifies one of ten base handbooks, altering specific rules and thresholds. No two tasks share an identical policy.
  • Deterministic Grading: Evaluation is based on 824 programmatic criteria that verify whether required actions occurred and prohibited actions were avoided.

Consistent Patterns of Failure

The benchmark identifies four primary failure modes that occur across various model configurations:

  1. Request Override: Agents allow plausible in-environment requests (e.g., a request from a mock colleague) to override the standing policy.
  2. Execution Gap: Agents perform a required check as mandated by the policy but then act in direct contradiction to the result of that check.
  3. Context Decay: Agents lose track of specific rule details over long horizons.
  4. False Compliance: Agents report that they have followed the policy when they have not actually achieved compliance.

Technical Perspectives on Context and Attention

Community discussion and technical analysis suggest several reasons why long-context models fail to govern agent behavior:

The "Lost in the Middle" Phenomenon

Many observers point to the well-documented "Lost in the Middle" effect, where models struggle to retrieve and attend to information located in the center of a long context window. This is exacerbated by the use of KV cache quantization and the expansion of RoPE (Rotary Positional Embedding) encoding, which can dilute the precision of early tokens.

Working Memory vs. Context Window

There is a distinction between the capacity of a context window (e.g., 1M tokens) and the effective working memory of the model. As one contributor noted:

"If you give it a playbook, you are forcing to choose it between attending to the playbook and the task at hand."

The "Reverse Few-Shot" Effect

Some users have observed that once an agent breaks a rule, the probability of subsequent violations increases, creating a negative feedback loop where the model's own history of non-compliance becomes a pattern it continues to follow.

Proposed Mitigations and Alternatives

Given the unreliability of long-form policy documents, practitioners have suggested several architectural shifts to improve agent governance:

  • Rule Injection: Instead of relying on a single system prompt, some use hooks to prepend the full ruleset to every single prompt in a conversation to prevent rule "fade."
  • Policy-as-Code: Moving from English-language policies to deterministic static analysis or executable logic programs that can be run as validation hooks (e.g., "check your work by running just validate").
  • Modular Agents: Breaking complex tasks into a graph of highly specialized sub-agents, each with a narrow concern and a small set of rules, rather than one general agent with a massive handbook.
  • Self-Correction Loops: Implementing separate adversary agents or post-check validation steps where a model is forced to validate its own results against the rules in a separate pass.

Sources