ScienceClaw – Self‑evolving AI agents for scientific work

What it is – ScienceClaw is an open‑source framework that lets a language‑model‑driven agent learn from its own verified executions. When the agent repairs a scientific workflow and that repair can be re‑played in a clean environment, the system extracts two things:

• a Skill (the high‑level strategy used to fix the problem), and
• an Operator (a typed, reusable piece of executable code). Both are stored together as a versioned update to the agent’s program. The update is only kept if it passes a replay check and improves performance on independent validation tasks.

---

🚀 Quick start (from the README)

# Clone and run the one‑click installer
git clone https://github.com/beita6969/ScienceClaw.git
cd ScienceClaw
chmod +x setup.sh && ./setup.sh   # installs Node, Python, weights, MCP servers, etc.

If you prefer a manual install you can use pnpm/pip and the provided CLI commands (scienceclaw.cli setup, doctor). The full install pulls ~19 GB of pretrained weights, but you can skip them (SCIENCECLAW_SKIP_TOOLS=1).

Bring‑your‑own model – the engine does not ship a specific LLM. Point it at any OpenAI‑compatible endpoint or a local executable:

export SCIENCECLAW_API_BASE_URL=<your endpoint>
export SCIENCECLAW_API_KEY=<your key>
export SCIENCECLAW_MODEL=<model name>

The config file packages/scienceclaw/configs/default.yaml lets you change the backend to a command‑line program or a Python factory.

---

🧠 How it works (high‑level)

1. Task definition – A task supplies inputs, an objective, a set of hard scientific constraints (units, feasibility, reproducibility, etc.), and a validation protocol.
2. Typed workflow graph – The agent builds a directed graph where each node is a tool or code snippet and each edge carries a schema (type, shape, unit, provenance). Nodes are fingerprinted so only the affected sub‑graph is re‑executed after an edit.
3. Replay‑gated verification – After a candidate repair, the whole graph is re‑run from a fresh environment. If the replay succeeds and the result respects all constraints, the repair is evidence.
4. Skill‑Operator extraction – The edit is split:
   • Skill patch – the high‑level planning change (e.g., “add a data‑cleaning step”).
   • Operator – the concrete executable component(s) that were added or modified, each with explicit input/output contracts.
5. Gate – The candidate persists only if:
   • the source replay reproduces the repair and the new components are actually used,
   • independent validation tasks show a strict improvement within a budget,
   • a noise guard (≥2 improved, ≤1 regressed validation runs) is satisfied. If any check fails, the incumbent program stays unchanged.
6. Versioning – Accepted updates become a new snapshot of the agent’s program. Snapshots can be promoted, inspected, or rolled back via the CLI.

---

📊 Results reported in the paper

• 23 disciplines (natural & social sciences) evaluated on 64 in‑distribution (IID) and 64 out‑of‑distribution (OOD) instances each.
• Mean OOD gain over a frozen baseline agent: +16.45 % (range +11.57 % to +23.34 %).
• OOD macro success rate rose from 77.72 % to 91.30 % after seven evolution rounds (+13.59 pp).
• Hard‑constraint pass rate: 98.23 % of instances.
• Promotion selectivity: 70 of 161 candidates (43.5 %) were accepted; the gate is selective but still allows steady improvement.
• Ablation studies show that the linked Skill‑Operator mechanism and the replay‑validation gate are the primary contributors to performance.

---

🔌 Using ScienceClaw from the OpenClaw gateway

Enable the plugin in ~/.openclaw/openclaw.json. Four new tools become available:

Tool	Operations
scienceclaw_canvas	open, act, render, replay, finish, status, list
scienceclaw_tools	search, show, status, weights, setup
scienceclaw_program	summary, skills, operators, show, history, rollback
scienceclaw_evolve	val_add, val_list, val_remove, propose, gate, run, status, candidates, show

Typical workflow (CLI example):

# 1. Declare a task
scienceclaw_canvas open '{"objective":"predict temperature","inputs":{...},"constraints":{...}}'

# 2. Let the agent build/repair the workflow
scienceclaw_canvas act   # one edit per round, feedback is typed

# 3. Verify the result
scienceclaw_canvas replay
scienceclaw_canvas finish   # returns the verified deliverable

# 4. Propose an evolution candidate
scienceclaw_evolve val_add <validation‑task>
scienceclaw_evolve propose   # extracts Skill+Operator
scienceclaw_evolve gate      # runs the replay + validation gate

# 5. Promote if you like it
scienceclaw_cli live promote <candidate-id>

A JSON‑RPC server (python -m scienceclaw.rpc) is also provided for programmatic control.

---

🛡️ Safety & governance

• Executable nodes run in a sandboxed container with static scans and runtime audit guards.
• No update is persisted without replay evidence, validation improvement, and budget compliance.
• All promotions are manual and versioned; you can roll back to any previous snapshot.

---

📦 What’s inside the repo

• packages/scienceclaw/ – core Python library, CLI, and RPC server.
• skills/ – >300 pre‑written skills; the initial seed includes 12 generic skills plus discipline‑specific ones (scienceclaw-benchmark-for30 … for52).
• extensions/scienceclaw/ – OpenClaw gateway integration.
• Benchmark data is hosted separately on Hugging Face (scienceclaw‑eval).
• Full design details are in packages/scienceclaw/docs/DESIGN.md and the accompanying paper (arXiv 2610.08691).

---

TL;DR

ScienceClaw is a real, research‑grade system that lets an LLM‑based scientific agent continuously improve by turning replay‑verified fixes into reusable, typed components. It provides a full CLI, a gateway plugin, and a benchmark suite covering dozens of scientific domains, and the authors report consistent out‑of‑distribution gains without changing the underlying language model.