stanford-iris-lab/meta-harness

What it solves

Meta-Harness addresses the problem of manually designing the "harness"—the surrounding code, memory systems, and retrieval logic that supports a fixed base model—to maximize its performance on specific tasks. Instead of retraining the model itself, this framework automates the search for the most effective code-based scaffold that helps the model succeed.

How it works

The framework implements an automated search loop that optimizes the code surrounding a frozen base model. It uses a "proposer agent" (such as Claude Code) to suggest and iterate on the harness code, which is then evaluated against a task. The system can be applied to various domains by defining a domain specification (domain_spec.md) and providing a wrapper for the proposer agent to log interactions.

Who it’s for

Researchers and developers who want to improve the model performance on a specific task without the cost of fine-tuning or retraining the model, specifically those working with agents, memory systems, and retrieval-augmented generation (RAG) scaffolds.

Highlights

• Automated Optimization: Shifts the focus from training the model to evolving the harness code.
• Extensible Framework: Includes an onboarding flow to help users apply the optimization loop to new domains.
• Reference Experiments: Provides implementations for text classification memory systems and scaffold evolution for Terminal-Bench 2.0.
• Agent-Agnostic: While examples use Claude Code, it supports other proposer agents via custom wrappers.