Anthropic Postmortem: Infrastructure Bugs Affecting Claude Response Quality

Between August and early September 2025, three distinct infrastructure bugs intermittently degraded the response quality of Claude. Anthropic has since resolved these issues and confirmed that model quality is never intentionally reduced based on demand, time of day, or server load.

Three Overlapping Infrastructure Issues

Anthropic identified three separate bugs that overlapped in timing, making diagnosis complex. The degradation was exacerbated on August 29 by a load balancing change that increased the volume of affected traffic.

1. Context Window Routing Error

A routing bug introduced on August 5 caused some Sonnet 4 requests to be misrouted to servers configured for the 1M token context window.

  • Impact: Initially affecting 0.8% of requests, the issue peaked at 16% of Sonnet 4 requests on August 31 following a load balancing change. Approximately 30% of Claude Code users experienced at least one misrouted message during this period. Impact on Amazon Bedrock peaked at 0.18%, while Google Cloud's Vertex AI was affected by less than 0.0004% of requests.
  • Persistence: Due to "sticky" routing, users who had one request misrouted were likely to have subsequent follow-up messages routed to the same incorrect server.
  • Resolution: Routing logic was fixed by September 4, with full rollout completed across first-party platforms, Vertex AI (September 16), and AWS Bedrock (September 18).

2. Output Corruption

A misconfiguration deployed to Claude API TPU servers on August 25 caused errors during token generation due to a runtime performance optimization.

  • Impact: This bug assigned high probabilities to tokens that should rarely be produced, resulting in unexpected characters (e.g., Thai or Chinese characters in English prompts) or syntax errors in code. It affected Opus 4.1, Opus 4, and Sonnet 4 requests on the Claude API between August 25 and early September. Third-party platforms were not affected.
  • Resolution: The change was rolled back on September 2, and detection tests for unexpected character outputs were added to the deployment process.

3. Approximate top-k XLA:TPU Miscompilation

A code deployment on August 25 intended to improve token selection triggered a latent bug in the XLA:TPU compiler.

  • Impact: The bug affected Claude Haiku 3.5 and potentially a subset of Sonnet 4 and Opus 3 on the Claude API. Third-party platforms were not affected.
  • Resolution: The bug was rolled back for Haiku 3.5 on September 4 and Opus 3 on September 12. Sonnet 4 was also rolled back as a precaution.

Technical Deep Dive: The XLA Compiler Bug

The XLA compiler bug involved a failure in the "approximate top-k" operation—a performance optimization used to find the highest probability tokens during text generation.

Precision Mismatch and Token Dropping

Claude's models compute probabilities in bf16 (16-bit floating point), but the TPU vector processor is fp32-native. The XLA compiler optimizes runtime by converting some operations to fp32 (32-bit) via the xla_allow_excess_precision flag. This created a precision mismatch where different operations did not agree on the highest probability token, occasionally causing the most probable token to be dropped entirely when temperature was zero.

The Failure of Approximate Top-k

In August 2025, a rewrite of the sampling code removed a December 2024 workaround that had been masking the underlying compiler bug. This exposed the approximate top-k operation, which returned completely wrong results for specific batch sizes and model configurations. Because the bug's behavior was inconsistent and depended on unrelated factors (such as preceding operations or enabled debugging tools), it was difficult to reproduce.

Final Resolution

Anthropic transitioned from approximate to exact top-k and standardized additional operations on fp32 precision. The company accepted a minor efficiency impact to ensure model quality.

Challenges in Detection and Remediation

Several factors contributed to the delayed detection of these bugs:

  • Evaluation Gaps: Standard benchmarks and safety evaluations failed to capture the degradation because Claude often recovers from isolated mistakes.
  • Privacy Constraints: Internal security controls limit engineer access to user interactions, preventing the reproduction of bugs using actual problematic interactions.
  • Noisy Signals: The overlapping nature of the bugs and their varying impact across platforms created a confusing mix of reports that appeared as random degradation.
  • Over-reliance on Evaluations: A spike in negative reports on August 29 was not immediately linked to a routine load balancing change.

Future Preventative Measures

Anthropic is implementing the following changes to prevent similar infrastructure failures:

  • More Sensitive Evaluations: Development of new evaluations that can more reliably differentiate between working and broken implementations.
  • Continuous Production Monitoring: Running quality evaluations continuously on live production systems to catch errors like the context window routing bug.
  • Enhanced Debugging Tooling: Creating infrastructure to debug community-sourced feedback more efficiently without compromising user privacy.

Sources

Related