What a Harness Actually Is, in 190 Lines of Code

1. The model is strong. The job still doesn't get done.

Anthropic ran a controlled experiment: same model (Opus 4.5), same task (build a retro 2D game maker), two runs.

The first let the model go solo — 20 minutes, $9. In the author's words: "My entities appeared on screen but nothing responded to input." The second wrapped it in a planner + generator + evaluator structure — 6 hours, $200: "I was actually able to move my entity and play the game."

Their own summary: "The harness was over 20x more expensive, but the difference in output quality was immediately apparent."

Not a token of the model changed. What changed was the ring of stuff around it.

That ring is what people call a harness. And the moment you try to improve yours, the first question arrives: what exactly am I supposed to change? The agent I installed? Or an AGENTS.md in my repo? You have seen both answers, and both claim to be about harnesses.

2. The word points at two things — that's why it won't resolve

The trouble is in the definition itself. The most widely circulated one calls a harness "all the engineering infrastructure outside the model weights," then immediately offers Claude Code, Cursor and Codex as examples.

Both sentences are true. They are not describing the same layer:

• One layer is mechanism. How the loop — ask the model → execute → feed the result back — actually turns, when it stops, which tools the model can reach, what shape results come back in. mini-swe-agent, Claude Code and Codex all live here, and this layer is a program somebody else wrote.
• One layer is content. Instruction files, verification commands, progress notes, directory conventions — the things that actually flow through the mechanism and end up in front of the model. Some of it ships with the program; the rest is yours.

The two are not peers. They are slots and filling: the code in the mechanism layer cuts a number of slots, and content is what flows through them.

Separate them and a very common confusion dissolves — if Claude Code is itself a harness, what is left for me to engineer? I'm not writing Claude Code.

The answer is that the two layers belong to different people. The mechanism really isn't yours to change. But whatever you can change inside the slots lands directly in front of the model — writing AGENTS.md, settling on a verification command, keeping a progress file. That's all it is. So harness engineering isn't building a harness. It's filling in slots somebody else cut.

The metaphor needs one discount, though: the slots aren't empty. Both kinds of harness arrive with filling of their own. mini-swe-agent's ships in the repo as config/mini.yaml; off-the-shelf tools like Claude Code and Codex — install and go, internals sealed — bake the system prompt, the tool list and the truncation rules into the program. The real difference is whether you can see it: the first you can open, read and edit; over the second you can only layer your own. How much room each slot leaves varies enormously, and section 4 does that accounting slot by slot.

The OpenAI team that shipped a million lines with Codex — five months, "0 lines of manually-written code" — puts this half more bluntly. Describing what changed about the work, they write that a software engineering team's primary job is no longer to write code, but to "design environments, specify intent, and build feedback loops." Environment, instructions, feedback — three of the slots below. They weren't changing Codex. They were filling slots.

Which is also why concept articles can't teach you a harness: what a slot looks like only becomes visible in code.

The 190 lines below draw them for you.

3. Five slots, in 190 lines

The core of SWE-agent/mini-swe-agent is src/minisweagent/agents/default.py. The README calls it "some 100 lines"; at 25941c8 we counted 190 (171 without blank lines and comments). The number is off, the conclusion isn't: it's small enough to read end to end.

The whole loop is two methods. run() is a while True, and step() is a single line:

def step(self) -> list[dict]:
    """Query the LM, execute actions."""
    return self.execute_actions(self.query())

Ask the model, run what it asked for. That is the entire trunk of an agent.

How does the loop stop? Look closely — this is the single most worth-stealing thing in the design:

if self.messages[-1].get("role") == "exit":
    break

The stop condition is not "the model said it was done." It is that the last message in the list carries the role exit. The model has no way to write such a message itself — it can only trigger some concrete event and have something else write it on its behalf. Who holds that authority? Sections 4 and 5 answer half each.

What gets bolted onto this skeleton next is a borrowed decomposition: split a harness into instructions, tools, environment, state and feedback. It comes from lecture 2 of learn-harness-engineering.

All five point at real line numbers inside these 190. Each section below says in one sentence what question the slot answers, then points it out in real code — concept articles usually do only the first half, and the shape of a slot only shows up in the second.

Slot 1 · Instructions

Instructions answer: what does the model know before it opens its mouth. They are the part of what you say that doesn't change from step to step — who it is, what rules bind it, what the job is, what counts as done. Of the five slots this is the only one made purely of words, which is exactly why it gets mistaken for the whole harness.

At the top of run(), two templates are rendered into the first two messages:

self.add_messages(
    self.model.format_message(role="system", content=self._render_template(self.config.system_template)),
    self.model.format_message(role="user", content=self._render_template(self.config.instance_template)),
)

system_template and instance_template are configuration, not code. They live in config/mini.yaml, and you can change them.

The templates carry placeholders that need values from elsewhere. One line of the instructions reads {{system}} {{release}} {{version}} {{machine}}, and all four come from the environment — section 4 shows exactly where. Filling them uses Jinja2's StrictUndefined: one placeholder it can't fill and it raises on the spot.

That is not Jinja2's default. The default renders an unfillable placeholder as an empty string and carries on as if nothing happened. Here that means the model receives instructions with a sentence missing, starts work anyway, and you never get a signal. Choosing StrictUndefined is refusing that ending: if a piece of the instructions is missing, better that nothing runs at all.

Slot 2 · Tools

Tools answer: what can the model do to the world. Without them it can only produce text — so the size of this slot is the ceiling on what the agent can do.

outputs = [self.env.execute(action) for action in message.get("extra", {}).get("actions", [])]

Most agents hand the model a tool list: read_file, write_file, search, run_command… each with a declared name, parameters and types, and the model picks one per step.

mini-swe-agent's list has one entry: a tool called bash, whose parameter table is a single command. mini.yaml shows the model that "list" as-is:

Call the bash tool with your command as the argument:

• Tool: bash
• Arguments: {"command": "your_command_here"}

So the only thing the model can do in a step is say one shell command. To read a file, cat. To edit one, sed. To search, grep.

What this buys you: you never add a tool again. Whatever is installed on the machine, the model can use — jq, rg, git, python — with no parameter declaration and no error handling written on the harness side for each of them.

The cost is that the model has to genuinely know the command line. So mini.yaml spends 46 lines teaching it: heredocs for creating files, sed for replacing by line, nl for reading a file with line numbers. The complexity saved in the tool slot moves, unchanged, into the instruction slot. The total work is identical; only the address changed.

Slot 3 · Environment

Environment answers: where do the tool's actions land. The same rm -rf on your laptop and in a disposable container are two different events — this slot sets the blast radius.

Environment is a Protocol with three methods — Python's notion of an interface agreement: any class carrying those three methods counts as an Environment, with no inheritance and no registration anywhere. So the ... in the method bodies below isn't code I elided; the file genuinely has no implementation. An agreement fixes shape, not behaviour.

class Environment(Protocol):
    config: Any
    def execute(self, action: dict, cwd: str = "") -> dict[str, Any]: ...
    def get_template_vars(self, **kwargs) -> dict[str, Any]: ...
    def serialize(self) -> dict: ...

Each method does one job:

• execute — run a command, hand the result back. The action dict from slot 2 is what feeds it; the entire tool slot comes to rest in this one method.
• get_template_vars — the opposite direction: hand out the environment's own facts (operating system, version…) to fill placeholders in the instruction templates. This is the wire along which the environment reaches back into the instructions, and section 4 looks at it directly.
• serialize — export the environment itself as a dict, which ends up inside the trajectory from slot 4.

LocalEnvironment is the plainest possible implementation: subprocess.Popen(command, shell=True, ...) starts a fresh subshell, with a 30-second default timeout.

On timeout it kills the entire process group (killpg), not just that subshell. The reason is &: appending it to a command in a shell means "run this in the background, don't wait for it," and the command returns immediately. Models write this readily — python -m http.server & to start a server, then curl it on the next step. But processes spawned that way hang off the subshell; kill only the subshell and they are orphaned — no parent, but not dead, still running on your machine. killpg takes the whole group down together.

And those three methods are the whole agreement. Swap in an implementation that executes inside a Docker container and not one word of the agent's 190 lines changes. This is what "slot" means, literally: shape nailed down, filling up to you.

Slot 4 · State

State answers: what does the model still remember on the next call. The model itself is stateless; every call starts from zero, and anything you don't replay does not exist for it. So "what this agent remembers" was never a property of the model. It is a property of the harness.

self.messages: list[dict] = []

A list. That is all of the runtime state.

The other half of state has to outlive the process, and that's what save() is for: messages, cost, call count and config serialized into a JSON file — what people call the trajectory. Post-mortems, comparisons and accounting all read it.

What matters is where the call sits: in the finally of the try wrapped around step() inside run()'s loop. finally means "run this no matter how this block ends" — even mid-exception, save() has to finish first, and only then does the exception continue outward toward the except branches in run() that can catch it. finally is a stop along the way; it does not swallow the exception.

Elsewhere this would merely be good hygiene. Here it is required, because nearly every ending this agent has is an exception: the model declaring completion raises Submitted, the cost limiter raises LimitsExceeded, the clock raises TimeExceeded (section 5 covers those). Put save() on the normal-return path and the endings most worth having a record of are the ones that leave nothing behind.

Slot 5 · Feedback

Feedback answers: in what shape does the result get back in front of the model. It has to be processed, because context windows are finite and command output isn't — the question was never whether to handle it, only where and by what rule.

Results are not thrown straight back. They pass through a template first:

observation_template: |
  {%- if output.output | length < 10000 -%}
  { "returncode": {{ output.returncode }}, "output": {{ output.output | tojson }} }
  {%- else -%}
  { "output_head": ..., "output_tail": ..., "elided_chars": ..., "warning": "Output too long." }
  {%- endif -%}

A single find / can print hundreds of thousands of characters, and feeding that straight back eats the context window in one go. So past ten thousand characters only the first and last 5,000 survive, with a count of what was dropped standing in for the middle.

This has a proper name — context budget management: the window is only so big, and somebody has to decide who gets to occupy it. In larger harnesses it is often an entire apparatus: dedicated components, a model that writes summaries, tiered storage. Here the whole implementation is that {%- if -%}: one length threshold, head and tail kept.

More notable is that it lives in configuration, not in code. Want the model to see more? Raise 10000. No Python is involved — which, by section 2's split, makes the context budget content in this harness, not mechanism.

4. Whatever you write goes into one of these slots

Now look back at what those "harness best practices" articles tell you to do. Every item maps onto a slot.

mini.yaml's instance_template holds exactly what those articles want you to put in AGENTS.md — recommended workflow, hard constraints, the definition of done. Take this constraint:

Directory or environment variable changes are not persistent. Every action is executed in a new subshell.

That is true in the most literal sense: every execute starts a new subprocess, and wherever the previous command cd'd to, whatever it exported, the next one knows nothing about it. So it isn't warning the model to be careful. It is a faithful description of slot 3's implementation.

The instruction layer and the environment layer have to agree. Without that sentence the model would plausibly type cd src, then ls on the next step, believing it is looking inside src/ — when in fact it went back to the start and is looking somewhere else entirely, and won't notice. A model can only act inside the world its instructions describe; describe that world wrong and every step goes wrong in the same place, reliably.

A coincidence worth noticing: that instance_template in mini.yaml is exactly 96 lines (69 without blanks). And the OpenAI team tried the "one big AGENTS.md" approach and failed at it; what they landed on was treating AGENTS.md as the table of contents rather than the encyclopedia, "roughly 100 lines," with the detail pushed into docs/ and read on demand. One is an opening instruction to an agent, the other a repository entry point for an agent — unrelated concerns that converged on the same order of magnitude.

More interesting still is the wire between two slots. LocalEnvironment.get_template_vars() drops all of platform.uname() into the template variables, so mini.yaml can write:

{%- if system == "Darwin" -%}
<important>
You are on MacOS. For all the below examples, you need to use `sed -i ''` instead of `sed -i`.
</important>
{%- endif -%}

The environment described itself into the instructions. Run on macOS and the model receives different instructions than it would on Linux. This kind of detail never appears in a concept article, because it doesn't belong to any single subsystem — it lives in the seam between two of them.

The definition of "done" lives in a slot too

Those articles keep insisting: never let the agent declare itself finished. Here that principle is implemented as one concrete path.

The instruction layer tells the model that finishing means running a particular command:

Submit your changes and finish your work by issuing the following command: echo COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT.

And the environment layer sniffs stdout after every execution:

lines = output.get("output", "").lstrip().splitlines(keepends=True)
if lines and lines[0].strip() == "COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT" and output["returncode"] == 0:
    raise Submitted({"role": "exit", ...})

The first line has to be exactly that magic string, and the return code has to be 0. Only then does it raise Submitted — and Submitted carries precisely that role: "exit" message. The stop condition from section 3 closes here.

So "the model can't declare itself done" means, in code: all the model can do is emit a shell command; that command has to genuinely succeed before the environment will write the exit message on its behalf. The verdict belongs to the environment, not to the model.

Your share, slot by slot

All of the above was said inside mini-swe-agent, where every slot's filling lies open in mini.yaml and you can move any of it. With an off-the-shelf tool like Claude Code the filling is sealed inside the program, and how much of each slot you can move varies a great deal:

Slot	Mechanism (fixed by the program)	Content (supplied by you)
Instructions	when they're injected, which files are read, what overrides what	AGENTS.md / CLAUDE.md, skill bodies, the request you're making now
Tools	the list, the calling protocol, the permission model	MCP servers, scripts a skill carries, and what's installed on the machine
Environment	subprocess on this machine, or inside a container	which one you pick, and what's in it — network, writable database
State	the message list, context compaction, which memory file is read	what goes into the progress doc, and at what granularity
Feedback	truncation rules, how results are fed back	what your commands print — especially the failing lines

Two rows deserve calling out.

The tools row looks least like yours, and isn't. As long as the list contains one shell tool, the content of that slot becomes "what is installed on your machine" — add rg, jq, gh and the model instantly has three more tools, with no change to the harness at all. That is the section-3 accounting in real life: mini-swe-agent took it to the extreme, while off-the-shelf tools simply mix a shell in among the rest.

Feedback's mechanism belongs to the agent; its content is almost entirely yours. You can't touch the truncation rule, but the text that gets fed back is produced by your repository. A test run that prints 1 test failed and one that prints an assertion diff with file and line put the model on completely different next steps; whether the typechecker exits silently or lists 12 errors is also your call. That is what OpenAI's "build feedback loops" refers to — they weren't changing Codex, they were making failure speak plainly.

5. Four limiters: why agents run away, and how to stop them

AgentConfig carries four limits, and they are the part of these 190 lines most worth stealing outright:

step_limit: int = 0
cost_limit: float = 3.0
wall_time_limit_seconds: int = 0
max_consecutive_format_errors: int = 3

The first three are checked inside query(), before the model is called:

if 0 < self.config.step_limit <= self.n_calls or 0 < self.config.cost_limit <= self.cost:
    raise LimitsExceeded(...)
if 0 < self.config.wall_time_limit_seconds <= int(time.time() - self._start_time):
    raise TimeExceeded(...)

0 < self.config.step_limit <= self.n_calls is Python's chained comparison, read as two conditions holding at once: step_limit > 0 (this limiter is switched on) and n_calls >= step_limit (it has been hit).

Which means setting a limit to 0 switches that limiter off — the left half, 0 < 0, is false immediately, and the right half is never even evaluated. It's a common sentinel convention: a config value has to be able to mean "no limit," and here the chosen value is 0.

Now read the defaults again: step_limit = 0, wall_time_limit_seconds = 0, cost_limit = 3.0. Out of the box only the cost limiter is live, capped at $3; steps are unlimited, and so is wall clock (real elapsed time).

The fourth limiter handles a different kind of failure: the model's output doesn't parse into an action. It doesn't die on the first one — max_consecutive_format_errors counts consecutive failures, and any clean step resets the counter to zero:

self.step()
self.n_consecutive_format_errors = 0  # reset on any clean step

The occasional glitch isn't a problem. Three in a row is a runaway.

And here is a detail you only get by reading the code. The format-error branch contains this line, with a comment attached:

# The call was billed before parsing failed, so query() never got to charge it.
self.cost += e.messages[0].get("extra", {}).get("cost", 0.0)

The model already spoke and the account was already charged; parsing simply failed before reaching the place that normally records it. Leave this line out and a model that keeps emitting bad format can burn money past the cost limiter indefinitely — because the cost never moves.

Those four limiters come to under ten lines together. They answer the question "why does the agent both overrun and never finish" — and the answer isn't that the prompt needed more work. It's that nobody gave it a termination condition.

6. So what is a harness?

Back to the word. Here is a formulation that won't jam on you again:

A harness is the ring of stuff around the model, and it comes in two halves. One half is mechanism: the program that runs the loop dictates which slots exist — instructions, tools, environment, state, feedback — and when it has to stop. The other half is the content flowing through those slots. The first half usually isn't yours to change; the second isn't entirely yours either, but everything you can change is in there.

What makes mini-swe-agent worth reading is that it compresses the first half into 190 lines, small enough to take in at once, so the slots become concrete things for the first time: instructions are two templates rendered at the top of run(), tools are env.execute(), environment is a three-method Protocol, state is a list plus one save(), feedback is a template with an if in it.

Once you can see where the slots are, you know where that AGENTS.md you wrote is actually going.

Sources

Every claim about code in this piece comes from SWE-agent/mini-swe-agent at 25941c8; the citations are listed one by one in the evidence bar at the top. External material:

• Anthropic: Harness design for long-running application development — the first-hand source for the paired figures in section 1.
• OpenAI: Harness engineering — leveraging Codex in an agent-first world — the source of two passages, in sections 2 and 4: the shift in what engineers do, and treating AGENTS.md as a table of contents rather than an encyclopedia at roughly 100 lines.
• Anthropic: Building effective agents — the source of the definition of an agent this piece works from ("LLMs using tools based on environmental feedback in a loop").
• walkinglabs/learn-harness-engineering — the five-subsystem split comes from lecture 2 of this course, and this piece follows it.

We also track this project in our directory: the mini-swe-agent page, with its star curve and a short summary.