Why Software Factories Fail: Harness Engineering Is Not Enough

Software factories that eliminate human code review fail because models cannot sustain codebase quality

The lights‑off software factory—where no human reads or writes code—does not work in practice. After trying a fully lights‑off approach in July 2025, the author’s team experienced outages, angry users, and had to dig into slop code that the agent had introduced. The promise that more loops, better harnesses, and more tokens would give both speed and quality ignores a fundamental limitation: today’s models are trained and evaluated only on whether a task passes tests, not on whether the resulting code stays maintainable over time.

Current training and benchmarks reward task completion, not maintainability

Model training uses reinforcement learning on traces that are scored by a simple pass/fail verifier (e.g., SWE‑bench Multilingual). There is no penalty for bad design, so models learn to add try‑catch blocks everywhere or use lazy type casts that undermine type systems. Because the cost of bad architecture appears weeks, months, or even years later, there is no fast oracle that can be used during RL to reward maintainability. As a result, even models that excel on frontier benchmarks can degrade a codebase over time, making simple changes require edits in many places and increasing the risk of hidden bugs.

Emerging benchmarks try to assess maintainability but remain limited

Efforts such as SWE‑Marathon (long, compound‑reward tasks), DeepSWE (large OSS tasks not in training data), and Frontier Code (multi‑PR tasks with mutation testing and a judge model) attempt to score qualities beyond pass/fail. However, a model that could reliably distinguish good code from bad would likely have written the good version initially, and no fast, reliable oracle for maintainability exists yet. These benchmarks are promising but do not yet provide a trustworthy signal for long‑term code health.

Reintroducing human review with front‑loaded leverage improves outcomes

Turning the lights back on—putting human review into the loop—combined with front‑loaded alignment reduces rework and makes review faster. The recommended process includes four phases: product review (short doc defining problem and success criteria), system architecture (visualizing services, endpoints, schemas, queues, stores), program design (defining types, method signatures, call‑stack trees, file‑tree diffs in pseudocode), and vertical slices (building end‑to‑end tracer bullets early and iterating). Spending about thirty minutes on this planning can cut review time from hours to minutes, as shown in the author’s experience.

Community insights reinforce the need for human judgment, governance, and modular design

Commenters highlighted several practical points:

• Governance and culture matter more than raw model skill; a thoughtful harness with human‑in‑the‑loop, security, and discipline enables a successful dark factory.
• Pull‑request user experience is poor; tools like Linear’s PR review feature that groups changes by theme and adds commentary reduce mental load.
• AI factories cannot manufacture intent; humans must supply the vision and guardrails.
• Building a modular architecture first allows agents to work on isolated modules where maintainability concerns are lower.
• Token subsidization may have driven early adoption of certain harnesses more than raw model quality.
• Optimizing the harness itself—using a representative repo‑level dataset to grade agent quality—can improve effectiveness without waiting for frontier model breakthroughs.
• Agents tend to copy patterns they see in the codebase, so establishing a clear, consistent pattern helps them generate usable code.
• In complex codebases, agents may repeatedly layer state variables; refactoring them into a sum type or using a formal model (e.g., in Quint) can restore clarity.
• A codebase over a year old with a backend and frontend still benefits from explicit human steering; the model alone cannot understand the code for the human.

Optimize within model constraints: keep humans in the loop for understanding and quality

The core constraint is that models are strong at generating code for isolated tasks but weak at preserving or improving codebase structure over time. Rather than chasing ever‑larger token counts, teams should accept this limitation, seek leverage through better planning and incremental review, and keep humans responsible for understanding intent, assessing architectural impact, and ensuring maintainability. This approach yields a realistic 2‑3× speedup with stable quality, avoiding the outages and technical debt that lights‑off factories produce.