Anthropic Shows Question Decomposition Boosts Faithfulness of Model-Generated Reasoning

TL;DR

Anthropic demonstrated that prompting large language models (LLMs) to decompose a question into simpler subquestions yields reasoning that more faithfully reflects the model’s internal process than traditional chain‑of‑thought (CoT) prompting, while retaining much of CoT’s performance gains.

Why Question Decomposition Matters

Decomposition forces the model to answer easier subquestions in separate contexts, which dramatically increases the alignment between the model’s stated reasoning and its actual computation. This higher faithfulness is crucial for verifying correctness and safety as LLMs tackle increasingly complex tasks.

Core Technical Approach

Decomposition Prompting: Instead of asking the model to produce a single, uninterrupted chain of reasoning, the prompt instructs the model to break the original query into a series of subquestions.
Separate Contexts: Each subquestion is answered in its own context, preventing the model from conflating steps or skipping reasoning.
Aggregation: The final answer is assembled from the subanswers, preserving the logical flow while keeping each reasoning step explicit.

Performance Relative to Chain‑of‑Thought

Decomposition‑based methods achieve strong results on standard question‑answering benchmarks, sometimes approaching the accuracy of CoT.
Crucially, on several recently proposed faithfulness metrics, decomposition outperforms CoT, indicating that the generated reasoning is a more truthful trace of the model’s internal decision‑making.

Faithfulness Metrics Evaluated

Anthropic evaluated the approaches using multiple metrics designed to assess how well the model’s written reasoning matches its actual computation. While the post does not list specific numbers, it reports consistent improvements across these metrics when using decomposition.

Implications for Model Safety and Verification

Enhanced Auditing: More faithful reasoning traces enable external auditors to inspect and validate model behavior step‑by‑step.
Safety Guarantees: When reasoning aligns with internal processes, it becomes easier to detect hallucinations, logical errors, or unsafe decision paths.
Future Research: The findings suggest a pathway toward reasoning that can be reliably used to certify model outputs, a key requirement for high‑stakes applications.

Limitations and Open Questions

The approach may incur additional computational overhead due to multiple context switches.
While faithfulness improves, the post acknowledges that performance does not always match the best CoT results, indicating a trade‑off that warrants further study.

Related Anthropic Research

Patterns and problems in emerging multi‑agent systems: Explores systemic risks in frontier models.
Reviewing the evidence on worker retraining programs: A policy‑oriented review co‑authored with external researchers.
Learning more about Claude's mathematical capabilities: Highlights advances in Claude’s ability to contribute to mathematical research, such as improving bounds related to the Riemann hypothesis.

This summary is based on Anthropic’s research announcement dated July 18 2023.