Did Claude Increase Bugs in rsync? A Statistical Analysis

In late May 2026, the rsync project became a flashpoint for the debate over AI-assisted coding after reports of regressions in recent releases were attributed to the use of Claude. However, a comprehensive distributional analysis of 36 rsync releases (v2.4.6 through v3.4.3) shows that Claude-assisted releases are statistically indistinguishable from historical releases in terms of bug density.

Statistical Findings: Claude vs. Historical Baseline

Claude-assisted releases do not exhibit an unusual bug rate when compared to the historical distribution of rsync releases. The analysis utilized a metric of severity-weighted bugs per 10 commits (sev/10c) to ensure that critical vulnerabilities were weighted more heavily than cosmetic issues.

  • No Statistical Abnormality: An exact permutation test yielded a p-value of 46%, meaning that if two releases were picked at random from the historical record, they would be as buggy or worse than the Claude releases 46% of the time.
  • Median Performance: A Fisher's exact test (p-value 74%) confirmed that Claude releases are no more likely to fall above the historical median bug rate than any other release.
  • Distributional Bracketing: The two Claude-assisted releases (v3.4.2 and v3.4.3) bracket the interquartile range (IQR) in opposite directions: v3.4.2 had zero real bugs, while v3.4.3 sat just above the IQR. Neither is a negative outlier.
  • Code Volume vs. Defect Rate: While Claude-assisted releases involved significantly more lines of code changed (averaging 3,756 lines vs. 696 for non-Claude releases, p=5%), the absolute number of severity-weighted bugs did not increase (p=77%).

Methodology and Severity Scoring

To avoid the pitfalls of simple bug counts, the analysis employed a structured approach to data collection and severity weighting.

Data Sources and Attribution

Bug reports were collated from GitHub issues, the rsync Bugzilla instance, and the rsync mailing list. Bugs were attributed to the most recent release shipped before the report, or to the specific version cited in Bugzilla.

Severity Weighting

Every bug report was scored on a 0–100 scale by Qwen 3 35B, acting as a senior reliability engineer. The rubric categorized bugs from "Data loss/corruption" (90–100) down to "Cosmetic/low impact" (10–29), with feature requests and spam scored as 0 and excluded from the bug counts.

Unit of Analysis

The release was chosen as the unit of analysis rather than the individual commit. This approach mirrors the claims made by critics—that the release became more buggy—and accounts for the fact that many bugs result from the interaction of multiple commits or are corrected by subsequent commits before the release is finalized.

Contextualizing the "Outrage"

The data suggests that the perceived decline in rsync quality was a narrative driven by social media rather than empirical evidence.

The Pre-Claude Outlier

Notably, the most buggy release in the entire dataset was v3.4.1—a release entirely prior to the introduction of Claude. It recorded 39.39 sev/10c (59 bugs in 9 commits). This release received no similar public outcry, suggesting that the presence of an "AI enemy" influenced the reaction to v3.4.3 more than the actual bug rate did.

The Causal Chain of Regressions

Technical discussion suggests that the increase in regressions was not caused by "vibecoding" but by a surge in AI-generated security reports. As maintainer Andrew Tridgell noted, a flood of AI-enabled CVE reports forced rapid, extensive changes to rsync's attack surface, which naturally increased the likelihood of regressions.

Community Counterpoints and Critiques

While the statistical analysis finds no evidence of harm, some community members raised concerns regarding the methodology and the nature of AI code:

  • Sample Size: Some critics argued that two data points (the Claude releases) are insufficient to draw a definitive conclusion.
  • Attribution Bias: Concerns were raised that bugs introduced in minor versions are often attributed to the longest-lived patch release, potentially skewing the data for v3.4.1.
  • Qualitative Decline: Some developers argued that quantitative metrics fail to capture the "sloppiness" of AI code, citing examples of nonsensical function renaming or poor NULL pointer handling that may not immediately trigger a bug but degrade long-term maintainability.
  • The "Rusty" Maintainer: A theory was proposed that agentic coding tools might make human maintainers "rusty" on fundamentals, leading to more edge-case errors over time.

"The trigger for the increased volume of changes (and hence increased number of regressions) was the influx of (mostly) LLM-enabled security issues. i.e. the causal chain was: LLMs $\rightarrow$ more known security issues $\rightarrow$ more changes needed than usual $\rightarrow$ more regressions than usual." — jbert on Lobsters

Sources