aisa-group/PostTrainBench

PostTrainBench

What it is – A benchmark that tests how well autonomous LLM‑based agents can post‑train (fine‑tune) a pre‑trained large language model to improve its performance on a downstream task. The agent is given a fixed compute budget (10 h on an H100) and an evaluation script, and its success is measured by the final score of the fine‑tuned model on the target benchmark.

Why it matters – As LLMs become more capable, the next frontier is letting other LLMs run the research loop: data collection, hyper‑parameter search, model updating, and evaluation. PostTrainBench provides a concrete, reproducible setting to compare different “LLM‑as‑research‑assistant” pipelines.

Key components

• Four CLI scaffolds – Claude Code, Codex CLI, Gemini CLI, and OpenCode. Each scaffold wraps a different LLM provider, letting the same benchmark be run with different agents.
• Six diverse downstream tasks – AIME 2025 (hard math), Arena Hard Writing (creative writing), GPQA (graduate‑level science), GSM8K (grade‑school math), HealthBench Easy (medical reasoning), and HumanEval (code generation). This breadth tests reasoning, domain knowledge, and coding ability.
• Reference execution environment – An HTCondor‑based HPC pipeline (the default) and a cloud‑GPU adapter (src/harbor_adapter/) that runs the whole benchmark on Modal without a cluster.
• Prompt template – A detailed system prompt (shown in the README) that tells the agent to autonomously research, experiment, and fine‑tune the model while respecting constraints (no test‑data leakage, limited time, only fine‑tune the specified base model, etc.).

How to run it

1. Install system dependencies (apptainer, fuse‑overlayfs).
2. Build the provided container (bash containers/build_container.sh standard).
3. Pull the HuggingFace cache needed for model loading.
4. Fill in an .env file with your API keys (OpenAI, Anthropic, Gemini, OpenCode, Z.AI) and optional paths.
5. Submit jobs with bash src/commit_utils/commit.sh (HTCondor) or, for cloud GPUs, follow the instructions in src/harbor_adapter/README.md.

Supported agents

• API‑based (OpenAI, Anthropic, Gemini, OpenCode, Z.AI) – keys are read from the environment and injected into the container.
• Subscription‑based CLI agents – Codex (ChatGPT Pro) and Claude Code (Claude Max) require a one‑time login/auth token that is copied into the job directory at submission time.

Project layout

Folder	Purpose
agents/	Implementations of the four CLI agents and their configuration files.
containers/	Docker/Apptainer definition and scripts to download a pre‑populated HuggingFace cache.
src/	Core benchmark code: job submission utilities, baseline calculators, evaluation task definitions, and the Harbor cloud‑adapter.
results/	Where each run writes its final scores (baseline runs are prefixed with baseline_).
logs/	HTCondor log files (git‑ignored).

Typical workflow

1. Choose a base LLM (e.g., llama‑2‑7b).
2. Pick a target benchmark (e.g., GPQA).
3. The agent receives the prompt, runs evaluate.py to get a baseline score, then iteratively gathers data, fine‑tunes, and re‑evaluates until the 10‑hour budget expires.
4. The best fine‑tuned checkpoint is saved in final_model/ and its score is recorded.

Who might use it

• Researchers studying LLM‑driven AI‑R&D and automated fine‑tuning pipelines.
• Teams building LLM agents that need a standard testbed for self‑improvement capabilities.
• Benchmark designers looking for a multi‑task, compute‑constrained evaluation suite.

Contact & citation

• Maintainers: Ben Rank, Hardik Bhatnagar, Maksym Andriushchenko (emails provided in the README).
• Cite the accompanying arXiv paper (arXiv:2603.08640).