Human-Agent-Society/reef

What it solves

Reef provides infrastructure for creating self-improving AI agents. It solves the problem of manually updating models or agent configurations when new data or feedback arrives, by automating the cycle of serving, observing feedback, training updates, and deploying them without interrupting service.

How it works

Reef operates as a continual learning backend that uses a four-step cycle:

1. Serve: It provides OpenAI- and Anthropic-compatible HTTP endpoints to serve agent requests and record every interaction.
2. Observe: It matches feedback (scores or text) reported by users or verifiers to the recorded interactions.
3. Grow: It uses "recipes" to produce updates based on eligible records. These updates can target either model weights or the agent's "harness" (rules, skills, prompts, and extensions).
4. Commit: It evaluates candidate updates against the current version and publishes them to a version history if they perform better.

Who it’s for

Developers building autonomous agents that need to improve their performance over time through real-world experience and feedback loops.

Highlights

• Dual-Surface Learning: Can evolve both the underlying model weights and the high-level agent harness (skills and prompts).
• Zero-Downtime Updates: Synchronizes updated weights and configurations to the serving runtime without requiring a restart.
• Standardized API: Uses standard HTTP endpoints for inference and feedback reporting, making it easy to integrate into existing workflows.
• Recipe-Based Architecture: Includes a cookbook of recipes for different workloads, such as tasks scored by tests or agent traffic with next-state signals.