XYBench: Evaluating AI Performance on the XY Problem
Overview
XYBench is a benchmark designed to evaluate how AI assistants handle the "XY problem," a scenario where a user asks for help with a specific, often misguided approach (Y) rather than the actual underlying goal (X). Developed by researchers at the Allen Institute for AI (AI2), Carnegie Mellon, NVIDIA, and Johns Hopkins, the benchmark consists of over 8,000 questions spanning programming, cooking, and home repair. It aims to address a critical gap in LLM performance: the ability to recognize implicit misconceptions and prioritize solutions that resolve the user's true intent.
The XY Problem in AI Assistance
The XY problem occurs when a user assumes a particular approach will work and asks for assistance with that approach, failing to realize the approach itself is flawed for their specific goal. For example, a user might ask how to thicken frosting that is sliding off a cupcake, when the actual solution is to wait for the cupcake to cool. Effective AI assistants must be capable of identifying these misconceptions and steering the user toward the correct solution, rather than simply answering the literal, flawed question.
Benchmark Construction
XYBench comprises 8,115 questions sourced from four distinct areas to ensure a broad range of real-world scenarios:
- Stack Overflow (1,557 questions): Real-world technical queries where the accepted human answer addresses the underlying goal.
- Stack Exchange (184 questions): Additional community-driven problem-solving data.
- Manual Curation (102 questions): Expert-selected examples of the XY problem.
- WikiHow (6,272 questions): Procedural guides generated to include plausible misconceptions, refined to ensure sufficient context for inferring intent.
Evaluation Metrics
To assess how well models handle these scenarios, XYBench measures three specific behaviors using Claude Sonnet 4.5 as an automated judge:
- Pragmatic Solution: Determines if the model provides solutions for both the stated request (Y) and the intended request (X).
- Pragmatic Emphasis: Evaluates whether the model focuses on the stated request, the intended request, or balances both.
- Misconception Identification: Measures the model's ability to steer the user toward the underlying goal while discouraging the mistaken approach.
Human validation of the automated judge showed high agreement, with 84.6–93.3% alignment on misconception identification and 74.2–79.8% on pragmatic emphasis.
Experimental Findings
Testing on frontier models—including Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro—revealed a consistent tendency to prioritize the stated request over the underlying goal.
- Emphasis Gap: While models frequently provided a solution to the underlying goal, they often buried it, focusing primarily on the mistaken approach. On Stack Overflow, models focused on the underlying goal in only 35–41% of responses.
- Recognition vs. Execution: A key finding was the gap between recognition and production. When presented with multiple-choice options, models successfully identified the goal-focused answer 71–84% of the time. However, when asked to generate free-form responses, they struggled to make that same goal the focus of their answer.
- Oracle Testing: Even when explicitly provided with the intended goal (X) alongside the stated request (Y), models failed to consistently outperform human baselines in redirecting users, suggesting that detecting intent is only one part of the challenge; models must also be trained to act on that intent.