CursorBench 3.1: Evaluating AI Coding Agents on Real-World Tasks

CursorBench 3.1 is a benchmark designed to evaluate AI coding agents on ambiguous, multi-file tasks derived from actual Cursor user sessions. The latest results indicate that Fable 5 Max currently leads the leaderboard in success rate, while Cursor's internal model, Composer 2.5, shows high efficiency at a significantly lower cost than frontier models.

CursorBench 3.1 Performance Rankings

Fable 5 models currently dominate the top of the leaderboard, with Fable 5 Max achieving the highest success rate at 72.9%. The benchmark highlights a significant gap between the top-performing Fable models and other frontier models like GPT-5.5 and Opus 4.8.

Top Performing Models

Rank	Model	Success Rate	Avg Cost / Task	Tokens Used	Steps
1	Fable 5 Max	72.9%	$18.02	63,842	76
2	Fable 5 Extra High	72.0%	$13.74	48,754	63
3	Fable 5 High	70.6%	$10.81	37,173	54
4	Fable 5 Medium	69.8%	$8.27	28,507	47
5	Opus 4.7 Max	64.8%	$11.02	62,989	96
6	GPT-5.5 Extra High	64.3%	$4.37	17,905	46

Efficiency and Cost Analysis

Composer 2.5 stands out as a highly cost-effective option, achieving a 63.2% success rate with an average cost of only $0.55 per task. This is a fraction of the cost of Fable 5 Max ($18.02) or Opus 4.7 Max ($11.02), despite maintaining a success rate competitive with several higher-cost models.

Benchmark Evolution and Methodology

CursorBench has evolved from version 3.0 to 3.1 to better reflect the complexities of professional software engineering.

• CursorBench 3.0: Focused primarily on edit, refactor, and bugfix problems.
• CursorBench 3.1: Expanded to include problems focused on codebase understanding, bugfinding, planning, and code review, while improving grading criteria for edit tasks.

Average cost per task is calculated using published per-million-token pricing (including input, cache read, cache write, and output) applied to the tokens used by each model on the benchmark tasks.

Community Critique and Technical Debate

While the benchmark provides a structured view of performance, the results—particularly those regarding Composer 2.5—have met with skepticism from the developer community.

Discrepancies with Third-Party Benchmarks

Critics point to a significant gap between CursorBench results and independent evaluations. One user noted that while Composer 2.5 performs well on Cursor's own benchmark, it lags significantly in others:

"You look at the DeepSWE benchmark... and GPT-5.5 xhigh gets a 64, Opus 4.8 max gets 56, and Cursor 2.5 gets 16."

Model Reasoning and Implementation Quality

Users have reported that while Composer 2.5 is efficient at executing plans, it may lack the critical reasoning capabilities of frontier models. Observations include:

• Reasoning Gaps: Some users find Composer 2.5 lacks the "thinking about the problem and critical reasoning" found in GLM 5.2 or Opus 4.6.
• Implementation Risks: Fable models, despite high scores, have been described as occasionally leaving "big, dangerous holes hiding inside implementations" if not closely monitored.
• Speed vs. Quality: GPT-5.5 Extra High is praised for its speed and adaptive thinking, whereas Opus 4.8 Max is described as slower and more prone to "needlessly chew on everything."

Concerns Over Benchmark Bias

Several commenters questioned the validity of a company-run benchmark, suggesting that the tasks may be inadvertently or intentionally optimized for the company's own model. This sentiment is echoed by the belief that the only truly reliable metric is a user's specific workload: "The only benchmark you can trust is your actual workload!"