OpenAI Harness Engineering: Leveraging Codex for Zero-Manual-Code Development

OpenAI has successfully built and shipped an internal software product containing approximately one million lines of code without a single line being written manually by humans. By utilizing Codex agents, the team reduced development time to roughly 1/10th of what traditional manual coding would require, shifting the human role from writing code to designing the environments, specifications, and feedback loops that enable agent autonomy.

The "No Manually-Written Code" Experiment

Starting in August 2025, a small team of engineers used Codex (guided by GPT-5) to generate everything from the initial repository scaffold and CI configuration to application logic, documentation, and internal tooling. Over five months, the team grew from three to seven engineers, managing roughly 1,500 merged pull requests (PRs).

Key outcomes of this approach include:

  • High Throughput: An average of 3.5 PRs per engineer per day.
  • Comprehensive Generation: Agents produced not only product code and tests but also production dashboard definitions, evaluation harnesses, and the scripts used to manage the repository itself.
  • Human-in-the-Loop Steering: Humans remained involved to prioritize work, translate user feedback into acceptance criteria, and validate outcomes, but never directly contributed code.

Redefining the Engineering Role: From Coding to Scaffolding

In an agent-first workflow, the primary engineering bottleneck is not the agent's capability, but the specification of the environment. OpenAI found that progress stalled when the environment was underspecified, leading humans to focus on "depth-first" work: breaking high-level goals into smaller building blocks and creating tools that make the agent's work legible and enforceable.

Agent-to-Agent Workflows

Humans interact with the system primarily through prompts. To drive a PR to completion, Codex is instructed to:

  1. Review its own changes locally.
  2. Request additional agent reviews both locally and in the cloud.
  3. Iterate based on feedback until all agent reviewers are satisfied.

Over time, the team shifted nearly all review effort to agent-to-agent interactions, minimizing the need for human PR reviews.

Increasing Application Legibility for Agents

To reduce the burden on human QA, OpenAI focused on making the application's internal state directly legible to Codex. This allows the agent to reason about the system without human intervention.

  • Runtime Access: The app is bootable per git worktree, allowing Codex to launch isolated instances for every change.
  • UI Validation: By wiring the Chrome DevTools Protocol into the agent runtime, Codex can use DOM snapshots and screenshots to reproduce bugs and validate UI behavior.
  • Observability: Agents have access to a local observability stack, allowing them to query logs via LogQL and metrics via PromQL to meet specific performance targets (e.g., ensuring service startup completes in under 800ms).

Repository Knowledge as the System of Record

OpenAI discovered that monolithic instruction manuals are ineffective for agents because they crowd out task context and rot quickly. Instead, they implemented a "map" approach:

  • AGENTS.md as a Table of Contents: A short file (approx. 100 lines) provides pointers to deeper sources of truth.
  • Structured Documentation: A docs/ directory serves as the system of record, containing indexed design documentation, architecture maps, and quality grades for product domains.
  • Execution Plans: Complex work is captured in versioned execution plans with decision logs checked into the repository.
  • Mechanical Enforcement: Dedicated linters and a "doc-gardening" agent ensure the documentation remains synchronized with the actual code behavior.

Enforcing Architecture and "Taste"

To prevent architectural drift in a fully agent-generated codebase, OpenAI employs strict mechanical constraints rather than micromanaging implementation details.

Rigid Architectural Model

Each business domain follows a fixed layering system with strictly validated dependency directions: Types $\rightarrow$ Config $\rightarrow$ Repo $\rightarrow$ Service $\rightarrow$ Runtime $\rightarrow$ UI. Cross-cutting concerns (e.g., auth, telemetry) enter only through explicit "Providers."\n

Taste Invariants

Custom linters enforce "taste" and reliability requirements, such as:

  • Structured logging and naming conventions for schemas/types.
  • File size limits.
  • Boundary parsing (e.g., using Zod) to avoid "YOLO-style" data probing.

Managing Entropy and "AI Slop"

Full autonomy can lead to the replication of suboptimal patterns. OpenAI manages this through a continuous "garbage collection" process:

  • Golden Principles: Opinionated mechanical rules are encoded into the repository.
  • Automated Cleanup: Background Codex tasks regularly scan for deviations from these principles and open refactoring PRs, which are often automerged after a brief human review.

Current Capabilities and Future Unknowns

The system has reached a threshold where Codex can end-to-end drive a new feature. From a single prompt, the agent can validate the codebase, reproduce a bug (recording a video of the failure), implement and validate a fix (recording a video of the resolution), and merge the change after responding to feedback.

OpenAI notes that while this has worked for internal launch, it remains unknown how architectural coherence will evolve over several years or how the system will adapt as models become more capable.

Sources