leo-lilinxiao/codex-autoresearch

codex‑autoresearch – Autonomous, measurable experimentation for OpenAI Codex

What it is – A Codex skill that lets the model run a closed‑loop “autoresearch” process on a Git‑tracked code base. You give Codex a numeric target (e.g., reduce error_count to 0). The skill then:

1. Inspects the repository and confirms the goal, the files it may edit, the metric command, and any regression guard.
2. Asks Codex to propose a single, focused change.
3. Commits the change, runs the verification command, and reads the metric.
4. Keeps the change if the metric improves (and the guard passes) or reverts it otherwise.
5. Repeats until the target is reached or a hard stop occurs.

All state is stored in an autoresearch-results/ folder (JSONL event log, HTML report, logs, etc.) and never gets staged in the repository.

---

Key Features

Feature	Why it matters
Git‑backed safety – every trial is a commit; failures are reverted with git revert.	Guarantees a reproducible, auditable history.
Metric‑driven loop – any command that outputs a single number (or a JSON with a numeric key) can be used as the objective.	Works for test failures, coverage, latency, binary size, security findings, etc.
Foreground & background modes – run interactively in the current Codex task or detach a worker that continues overnight.	Flexible for quick tweaks or long‑running optimisation.
Explicit confirmation – before the first edit Codex shows the goal, scope, metric, guard, and mode; you must approve.	Prevents surprise changes.
Rich reporting – immutable run.json, append‑only events.jsonl, and a self‑contained report.html visualising metric trajectory.	Easy to audit and share results.
Safety model – out‑of‑scope edits, malformed metrics, timeouts, or guard failures abort the run with a clear error.	Makes autonomous runs trustworthy.

---

Typical Use Cases

• Fix flaky tests – target error_count = 0 for a test‑suite runner.
• Reduce binary size – metric is the size reported by du -b.
• Improve performance – metric is latency measured by a benchmark script.
• Automate security hardening – metric is the count of static‑analysis warnings.
• Iteratively refactor – any measurable quality that can be expressed as a single numeric value.

---

Quick Installation & First Run

# Install the skill (requires a recent Codex release that supports skill‑installer)
skill-installer install https://github.com/leo-lilinxiao/codex-autoresearch

# Open a clean repository with full write access for Codex
codex --dangerously-bypass-approvals-and-sandbox

# Start an experiment (example: drive error_count → 0)
$codex-autoresearch
# You’ll be prompted with the baseline, target, scope, verify command, etc.
# Confirm and let Codex run in foreground or background.

For a manual install see docs/INSTALL.md.

---

How the Loop Works (Simplified)

inspect evidence → propose one change → commit & run metric →
   if improved & guard passes → keep commit
   else → revert commit
append event to events.jsonl → repeat until target reached

The control script owns the Git operations and state files; Codex owns the hypothesis generation and code edits.

---

Safety & Trust

• Every trial is a real Git commit, never an in‑memory edit.
• Non‑improving or guard‑failing trials are automatically reverted.
• The skill aborts on malformed metrics, command failures, timeouts, or when the repository drifts (e.g., branch change).
• No files are staged; all artefacts stay in autoresearch-results/.
• A run reports complete only when the retained metric meets the confirmed target.

---

Documentation & Help

• Installation – docs/INSTALL.md
• User Guide – docs/GUIDE.md (configuration, lifecycle, troubleshooting)
• Examples – docs/EXAMPLES.md (sample prompts, metric patterns)
• Contributing – CONTRIBUTING.md
• FAQ – built into the README (covers Git requirement, stopping/resuming, full‑access need, etc.)

---

License

MIT – see LICENSE.

---

Bottom line: codex‑autoresearch is a concrete, Git‑backed automation layer that lets OpenAI Codex run hypothesis‑driven code changes until a numeric goal is met, with full auditability and safety guarantees. It is a genuine software project aimed at autonomous experimentation on code bases, squarely within the AI‑assisted development frontier.