Human-Agent-Society/reef

Infrastructure for continually self‑improving agents

What it solves

Reef provides infrastructure to automate the continuous improvement of AI agents. Instead of static deployments, it creates a feedback loop where agents learn from their own interactions, allowing them to refine their performance over time without manual intervention.

How it works

Reef operates through a four-step cycle:

  1. Serve: It hosts the agent and records all interactions.
  2. Observe: It collects feedback (such as scores or corrections) and links it to specific agent responses.
  3. Grow: It uses "recipes" to process this feedback and generate updates, either by training new model weights or refining the agent's "harness" (prompts, rules, and skills).
  4. Commit: It evaluates these updates and publishes them to a versioned history, allowing the agent to improve while staying live.

Who it’s for

This project is for developers and researchers building autonomous agents who want to implement self-improving capabilities. It is particularly useful for those working on scientific discovery, task-stream learning, or agents that need to adapt based on real-world usage feedback.

Highlights

  • Dual-Surface Evolution: Supports both model weight training (using GPU stacks) and harness optimization (using model APIs).
  • Version Management: Automatically tracks and manages versions of agent updates, ensuring stability during the learning process.
  • OpenAI-Compatible: Provides an inference endpoint that is compatible with standard OpenAI and Anthropic API formats.
  • Recipe-Driven: Includes a catalog of pre-built recipes for different tasks, such as scientific discovery and continual learning.

Written about in

Related

  • Dispatch
    Raven: The Harness of Harnesses for Recursive Self‑Improvement

    Raven is an open‑source, memory‑first multi‑agent platform that orchestrates built‑in and third‑party agents, supports runtime self‑evolution, and demonstrates end‑to‑end project delivery and recursive self‑improvement benchmarks.

  • Project
    google-deepmind/reverb

    Reverb is a Linux‑only, Python‑friendly library (with a C++ backend) that provides a gRPC‑based, distributed experience‑replay service for reinforcement‑learning research. It lets you define tables with configurable sampling/removal strategies (uniform, prioritized, FIFO, etc.), rate limiters, and capacity limits, and supports checkpointing and sharding across multiple servers. Install via `pip install dm-reverb[tensorflow]` and use the `reverb.Server`/`reverb.Client` API to insert and sample data. Designed for research, not production, and tightly coupled to specific TensorFlow versions.

  • Project
    bytedance/deer-flow

    DeerFlow is an open‑source framework that lets you build autonomous AI agents capable of orchestrating sub‑agents, using tools (web search, sandboxed code execution, RAG), and managing memory. It supports many LLM providers, offers a web UI, a terminal UI, and integrates tracing services, all deployable via Docker.

  • Project
    Tencent/WeSmartFlow

    WeSmartFlow is an open‑source adaptive learning platform that uses AI agents to tutor, generate courses, and track progress in a personal knowledge graph, with a Vue front‑end, FastAPI back‑end, and optional WeChat integration.

  • Project
    WeaveMindAI/weft

    Weft is a compiled, graph‑based language and runtime for orchestrating AI workflows. It lets you connect LLM calls, agents, APIs, databases, and human‑in‑the‑loop steps as nodes in a typed graph, validates the design before execution, persists long‑running jobs, and records a journal of all data flows. Integrated with the AI assistant “Tangle”, it supports automatic code generation, testing, and versioned runs. Early‑stage but usable, with Docker‑based installer, VS Code extension, and examples like a WhatsApp support bot.