Measuring Code Sloppiness: Metrics, Findings, and Community Insights

TL;DR

LLM‑generated code can be formally correct yet significantly sloppier than human‑written code; simple metrics such as LOC change, Verbosity, and Erosion reveal that agents produce roughly double the verbosity and erosion of established repositories, and existing evaluation methods fail to catch this degradation.

Why Measuring Sloppiness Matters

Code that passes hidden tests can still contain unnecessary abstractions, duplicated logic, and poor architectural decisions. In large‑scale projects that add millions of lines of code (LOC) per month, this "slop" erodes human agency because developers cannot keep up with the hidden technical debt. The author argues that agents cannot autonomously resolve sloppiness, making quantitative evaluation essential.

Naïve Evaluation Approaches Fail

AI‑as‑Judge is Unreliable

Prompting an LLM to rate its own code on a 1‑10 scale behaves like a random number generator.
Pairwise comparisons ("A vs B") are unstable: renaming solutions can flip the model’s preference, as shown in arXiv:2604.16790.
Even sophisticated rubrics or LLM‑generated tests fall short of eliminating slop.

"Asking LLMs to judge the code they write is not a substitute for a proper evaluation." – Author

Human‑in‑the‑Loop is Not Scalable

Human review guarantees readability but cannot scale to the volume needed for training or benchmark suites.
The cost of reviewing millions of LOC outweighs the benefits of continuous ranking of model providers.

Simple Metric: LOC Change

Counting the net change in LOC after a code modification proved surprisingly effective at flagging sloppiness. The author cautions, however, that optimizing for this metric would trigger Goodhart’s law—once a metric becomes a target, it ceases to be a good measure.

Research‑Based Metrics from SlopCodeBench

The paper SlopCodeBench (arXiv:2603.24755v1) introduces two metrics that separate legacy codebases from LLM‑generated slop.

Verbosity

Measures duplicated or unnecessary verbose lines:

$$ \text{Verbosity} = \frac{|\text{AST‑Grep flagged lines} \cup \text{clone lines}|}{\text{LOC}} $$ Implemented via handcrafted heuristics in AST‑Grep.

Erosion

Quantifies how much code mass is concentrated in a few large, complex functions:

$$ \text{mass}(f) = \text{CC}(f) \sqrt{\text{SLOC}(f)} $$ $$ \text{Erosion} = \frac{\sum_{f:,\text{CC}(f)>10}\text{mass}(f)}{\sum_{f}\text{mass}(f)} $$ CC(f) is cyclomatic complexity; SLOC(f) is source lines of code.

Empirical Findings

Dataset	Verbosity (mean ± sd)	Erosion (mean ± sd)
Established human repositories	0.15 ± 0.06	0.31 ± 0.17
Code generated by agents (SlopCodeBench)	0.33 ± 0.10	0.68 ± 0.20

Agents produce roughly twice the verbosity and erosion of human code.

Additional anecdotal checks on the author’s own “vibe‑coded” projects showed Verbosity up to 0.4 and Erosion up to 0.75, confirming that the effect is not limited to the benchmark.

Why Agents Can’t Self‑Correct Slop

SlopCodeBench evaluates agents in an iterative setting: after each instruction‑test round the model’s context is erased, mimicking real‑world usage where code evolves over many steps.
Bad decisions accumulate, leading to a 0 % strict solve rate for state‑of‑the‑art models—i.e., no model passes all hidden tests at every checkpoint.
This stark failure warns against unbounded LOC growth powered solely by AI.

Community Reactions and Extensions

"The most important problems for sloppiness are global properties, not local ones. Architectural measures like separation of concerns are needed." – dherman

"Optimizing for minimum LOC may actually increase code‑golfing and produce one‑liners that are hard to maintain." – cjalmeida

"Metrics like cyclomatic complexity, churn, and authorship are already useful in internal dashboards; combining them could give a richer picture of sloppiness." – pbjerkeseth

"Goodhart’s law is invoked without discussion; we need to examine whether minimizing LOC truly leads to sloppier code." – drsopp

"Global architectural metrics (e.g., coupling, cohesion) are missing from the current discussion and may be essential for large codebases." – dherman

"Even with agents, human design work is still required; otherwise the codebase becomes a spaghetti of ad‑hoc features." – cheney_2004

These comments highlight three recurring themes:

Need for global architectural metrics (coupling, cohesion, layering).
Risk of metric gaming (Goodhart’s law) and the importance of multi‑metric suites.
Human oversight remains critical, especially for design and maintainability.

Open Directions

The author lists promising avenues for future research:

Coupledness of functions – measuring how tightly functions depend on each other.
Code churn – tracking rapid additions/removals as a proxy for instability.
Cohesion – assessing whether a module’s responsibilities are well‑focused.
Feedback loops – feeding Verbosity/Erosion scores back into LLM training to steer generation toward cleaner code (while monitoring for Goodhart effects).

Practical Takeaways for Practitioners

Track LOC change as a quick sanity check, but treat it as a heuristic, not a hard target.
Implement Verbosity and Erosion using AST‑Grep or similar static analysis pipelines to flag duplicated/over‑complex code.
Combine multiple metrics (complexity, churn, coupling) to mitigate the risk of any single metric being gamed.
Maintain human review cycles for architectural decisions; automated metrics can surface issues but cannot replace design judgment.
Design benchmarks that mimic iterative development (as SlopCodeBench does) to surface sloppiness that only appears after many refinement steps.

Conclusion

Quantitative metrics such as LOC change, Verbosity, and Erosion provide a concrete lens on code sloppiness, revealing that current LLM agents generate code that is roughly twice as verbose and eroded as human‑written code. While these metrics are valuable, they must be part of a broader, multi‑dimensional evaluation strategy that includes global architectural properties and human oversight to avoid Goodhart’s law and ensure maintainable software.