Does Code Cleanliness Affect Coding Agents? Analysis of arXiv:2605.20049

Code Cleanliness Does Not Impact Pass Rates but Reduces Operational Costs

Research indicates that the structural and stylistic quality of a codebase—its "cleanliness"—does not materially change the success rate of autonomous coding agents in completing tasks, but it significantly lowers the computational resources required to achieve those results. In a study involving 660 trials with Claude Code, agents working on cleaner code used 7% to 8% fewer tokens and reduced file revisitations by 34% compared to those working on "messy" versions of the same repositories.

Experimental Methodology: The Minimal-Pair Approach

To isolate the impact of code cleanliness from other variables like agent capability or project architecture, researchers employed a minimal-pair evaluation protocol.

Repository Construction

• Minimal Pairs: The study used repositories that matched in architecture, dependencies, and external behavior but differed in cognitive complexity and static-analysis rule violations.
• Generation: These pairs were created using agent pipelines that either degraded a clean repository or cleaned a messy one.
• Evaluation: 33 tasks were distributed across six such pairs, with success measured by hidden tests at the application's public surface.

Key Findings on Agent Efficiency

While the final outcome (pass/fail) remained consistent regardless of code quality, the process of reaching that outcome differed significantly:

• Navigational Efficiency: Agents in cleaner environments revisited files 34% less often, suggesting that well-organized code allows agents to locate relevant logic more quickly.
• Token Consumption: A reduction of 7-8% in token usage indicates that cleaner code reduces the amount of context the agent must process to understand the system.
• Conclusion: Traditional maintainability principles—such as modularization and clear naming—remain relevant in AI-driven development because they optimize the operational footprint and latency of coding agents.

Community Critique and Technical Counterpoints

Technical discussions following the study's release have highlighted several concerns regarding the experimental design and real-world applicability.

Concerns Over Experimental Rigor

Critics argue that the use of LLMs to synthetically "degrade" or "clean" codebases may not accurately represent real-world technical debt. Furthermore, some observers noted a significant gap in the evaluation process:

"Pass rate scores the agent’s final state against the hidden tests we wrote for each task. We do not check whether the agent broke unrelated tests already present in the repository... Any conclusions with respect to token consumption seems pretty meaningless if we're not controlling for the quality of the final output."

Real-World Performance Divergence

Many practitioners report a more substantial delta in performance than the study suggests, particularly when dealing with "dead code, redundant code, unreachable fallbacks, [and] leaking abstractions."\n

• Cognitive Load: Practitioners suggest that clean architecture and good encapsulation reduce the number of QA/review rounds required for frontier models to get a task right.
• Latency and Traversal: Beyond token counts, the "round trip latency" (queue time, prefill, decode) associated with searching through a spread-out or messy codebase significantly slows down task completion in production environments.
• Context Window Constraints: Because context windows are bounded and attention is inconsistent, locality and structure are viewed as critical for efficient agent operation.

Practical Strategies for AI-Native Codebases

Based on the synthesis of the study and practitioner insights, several strategies emerge for optimizing code for AI agents:

• Semantic Hinting: Organizing code into logically named files helps agents find relevant code in the first attempt, avoiding the need to read multiple chunks of bloated files.
• Refactoring for "Pythonic" or Idiomatic Standards: Prompting agents to refactor code to follow popular open-source organization standards can improve both agent performance and overall codebase maintainability.
• LSP Integration: Providing agents with access to a Language Server Protocol (LSP) can reduce the token cost of traversal by allowing the agent to jump directly to definitions rather than searching through files.