AMAP-ML/LongHorizon-Harness

LongHorizon‑Harness – Loop Engineering for Computer‑Use Agents

What it is

• A Python‑based framework that wraps existing LLM‑driven agents (Claude Code, Codex, OpenCode, DeepSeek Harness) with a loop that lets them run for hours on real desktop applications and the terminal.
• The loop consists of three roles that are executed each round:
  1. Manager – recomposes the goal, verified progress and any failure evidence, and plans the next bounded step.
  2. Executor – runs that step in a fresh context (GUI click/typing or CLI command).
  3. Auditor – independently inspects the filesystem, UI, logs or tests to verify the step actually succeeded.
• Only verified results become part of the persistent task state; failures are recorded as evidence for the next round.

Why it matters

• Existing code‑generation agents are good at a single‑turn prompt but break down on long, multi‑step computer‑use tasks. LongHorizon‑Harness supplies the missing execution, verification, and recovery machinery, enabling agents to complete complex workflows that span many applications.
• It works without retraining any model – you simply point the harness at the agent you already use.

Key features

• One‑command install (uv tool install lh-harness or pip install lh-harness).
• Multi‑backend support – Claude Code, Codex CLI, OpenCode, DeepSeek Harness, or any custom AgentAdapter.
• GUI + CLI coverage – can click, type, scroll, edit documents, run terminal commands, install packages, etc., all under a single task.
• Dashboard – a React/FastAPI web workbench (lh-harness web) for starting runs, answering approvals, sending mid‑run instructions, and inspecting logs.
• Plugin system – install a computer‑use plugin (codex-computer-use, open-computer-use, clawdcursor) that gives the agent OS‑level control.
• Robust recovery – evidence from failed rounds is fed back into the next planning step; runs can be paused, resumed, or continued with follow‑up queries.
• Role‑level configuration – each of Manager, Executor, Auditor can use a different model or backend.
• Diagnostics – lh-harness doctor checks the environment (Python, agent binaries, Node.js, permissions).

Benchmarks & results

Benchmark	Metric	Baseline (Claude Code)	LongHorizon‑Harness	Gain
WeaveBench (114 tasks)	Pass Rate	51.8 %	80.7 %	+28.9 %
WeaveBench	Overall score	0.702	0.835	+0.133
OSWorld 2.0 (108 tasks)	Binary success	2.8 %	8.3 %	3×
OSWorld 2.0	Partial success	21.5 %	35.2 %	+13.7 %
Terminal‑Bench 2.1	Success rate	69.7 %	77.2 %	+7.5 %
All experiments used Qwen 3.7‑Plus as the underlying model and Claude Code as the execution backend.

Typical workflow

1. Install the harness and a computer‑use plugin.
2. Run lh-harness init in your project to create a config.toml.
3. Start a run – either via the web UI (lh-harness web --workspace-root .) or directly from the CLI:

   TASK="Inspect the current directory and summarize its files."
   lh-harness run --task "$TASK" --agent codex
   

4. The harness will repeatedly:
   • plan the next step,
   • execute it in the chosen environment,
   • verify the outcome,
   • checkpoint progress or record failure evidence.
5. When the goal is reached, a plain‑language final answer is printed and a JSON report is saved under ./.lh-harness/runs/<run‑id>/.

Installation snapshot

# recommended isolated install
uv tool install lh-harness   # or: pip install lh-harness

# install a computer‑use plugin (example for Claude Code)
lh-harness plugin install open-computer-use

# initialise a project
cd /my/project
lh-harness init

Supported platforms

• Developed and tested on macOS. Windows support currently requires WSL2 (POSIX‑only primitives are missing).
• Requires Python ≥ 3.10, Node.js ≥ 20 for the plugins, and the chosen agent binary on PATH.

License

• MIT License (see LICENSE).

---

Bottom line – LongHorizon‑Harness is a practical, open‑source “loop engineering” layer that lets you turn any LLM‑based code/assistant agent into a reliable, long‑running computer‑use system capable of handling real‑world GUI and CLI workflows.