modem-dev/slop-scan

What it solves

It identifies "slop"—low-judgment, suspicious, or boilerplate patterns often found in unreviewed AI-generated code—within JavaScript and TypeScript repositories. It helps developers distinguish between mature, human-reviewed open-source software and codebases that feel "vibe-coded" or overly reliant on generated boilerplate.

How it works

The tool uses a deterministic CLI to scan repositories for specific heuristics (rules) that flag common AI-associated patterns. It provides raw and normalized scores (by file count, logical KLOC, and function count) to allow fair comparisons between repositories of different sizes. It also supports a delta mode to track whether these patterns are increasing or decreasing over time or across different branches.

Who it’s for

• Developers auditing third-party repositories for quality.
• Teams wanting to find low-judgment boilerplate in their own codebases.
• Researchers comparing AI-generated repositories against mature OSS baselines.
• Users iterating on deterministic heuristics for detecting generated code.

Highlights

• Deterministic Scanning: Uses a set of tested rules to find patterns like empty catch blocks, error-obscuring catch blocks, and duplicated function signatures.
• Normalized Metrics: Compares repos using ratios (e.g., Score/KLOC) to ensure fair benchmarking.
• Extensible: Supports third-party rule plugins and custom configuration via slop-scan.config.json.
• CI/CD Integration: Offers a --lint mode for human-readable logs and a --json mode for machine-readable output to fail builds based on new findings.
• Pinned Benchmarks: Includes a benchmark set comparing AI-generated repos against mature OSS pinned to commits from before 2025 to establish a baseline.