yu-xin-c/Sea-mult-agent

What it solves

Sea-Mult-Agent (ScholarAgent) is a multi-agent system designed to automate scientific research tasks, specifically focusing on paper reproduction, benchmarking repositories with custom data, and conducting budget-constrained automated research (AutoResearch). It addresses the challenge of manually iterating through code patches, hyperparameters, and method configurations to find optimal results within a fixed resource budget.

How it works

The system uses an Intent Router and Planner to convert user goals into a validated Directed Acyclic Graph (DAG) of tasks. A Scheduler then routes these tasks to specialized agents:

• Librarian: Extracts claims and methods from papers and freezes a reproduction rubric.
• Coder & Research Coding Agent: Handles repository discovery, dependency management, code debugging, and generating code patches or adapters for custom data.
• Benchmark Agent: Manages data splitting (train/val/test), leak checks, and calculates final metrics using hidden labels to prevent overfitting.
• Sandbox: A deterministic Go-based service that executes code in isolated Docker containers with persistent workspaces.
• Research Optimizer: A Python-based component that uses UCB (Upper Confidence Bound) and UCT-style tree searches (Beam search) to allocate budget across parameter trees and select high-value candidates.

Who it’s for

It is primarily for researchers and developers who need to automate the reproduction of AI papers, evaluate existing repositories on their own proprietary datasets, or perform systematic ablation studies and hyperparameter optimization under strict time or resource constraints.

Highlights

• Budget-Constrained AutoResearch: Implements a two-layer Tree of Thoughts (ToT) and UCT-style search to optimize methods and hyperparameters within a fixed budget.
• Isolated Execution: Uses a dedicated Go sandbox service to run experiments in native Docker containers, ensuring reliability and rollback capabilities.
• Claim-to-Evidence Graph: Binds paper claims to actual execution artifacts and metrics, creating a visual evidence map for reproduction verification.
• Hidden Holdout Validation: Prevents "cheating" by using a Benchmark Agent that calculates final metrics on labels hidden from the research agents.
• Integrated Workbench: Provides a React-based UI for monitoring DAG execution, viewing real-time logs via SSE, and analyzing experimental ledgers.