huggingface/Repo2RLEnv

Repo2RLEnv – Turning Code Repositories into Verifiable RL Tasks

What it is – Repo2RLEnv is a Python library (and CLI) that automatically creates Harbor reinforcement‑learning environments from existing software artifacts: merged pull requests, commit histories, CVE patches, terminal recordings, etc. Each generated task contains an instruction, a starting container, a private verifier (the reward function) and a reference solution. The tasks follow the open‑source Harbor task format, so any Harbor‑compatible agent can be trained or evaluated on them.

Why it matters – Training coding agents (e.g., code‑completion, bug‑fix, or code‑generation models) requires thousands of realistic programming problems with an automatic way to check success. Manually building such environments is slow and error‑prone. Repo2RLEnv harvests real development data and turns it into ready‑to‑use RL environments, dramatically lowering the cost of creating high‑quality benchmarks.

Key features

• 23 generators covering repository repair, implementation, patch‑reproduction, terminal‑state tasks, and pure reasoning/optimization problems.
• 21 published datasets (≈1,930 tasks) on the Hugging Face Hub, ready to download.
• Tasksmith – an agent that can automatically build new tasks from a PR, using LangGraph orchestration and remote workers (Modal or Daytona).
• Harbor compatibility – tasks can be run, reviewed, and shared with any Harbor agent via the harbor CLI.
• Zero‑LLM‑key quickstart – the simplest pipelines run locally with only Python 3.12+ and the GitHub CLI.

Typical workflow

1. Generate a set of tasks from a repo (e.g., recent PRs of pallets/click).
2. Validate the tasks locally with the built‑in oracle to ensure the reference solution scores 1.0.
3. Run an agent (e.g., Claude‑Sonnet) against the tasks using harbor run.
4. Publish your own generated tasks back to the Hub with a single command.

Installation

# Core library and CLI
pip install repo2rlenv

# Add Harbor support (needed to run/review tasks)
pip install 'repo2rlenv[harbor]'

# Full stack with Tasksmith on remote workers (Modal/Daytona)
pip install 'repo2rlenv[tasksmith,daytona,harbor]'

Quickstart (run in ~5 min, no Docker, no LLM API key)

pip install 'repo2rlenv[harbor]'
gh auth login

# Generate three PR‑diff tasks from the click library
repo2rlenv generate --repo pallets/click \
  --pipeline pr_diff --pipeline-opt limit=3 --out ./tasks

# Verify they are correct
repo2rlenv validate ./tasks --oracle

# Train/evaluate an agent (example uses Anthropic Claude)
harbor run -p ./tasks -a claude-code \
  -m anthropic/claude-sonnet-4-6 \
  --ae ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY --env docker

Tasksmith in a nutshell – Point Tasksmith at a merged PR and a coding agent; it will:

1. Analyse the change and repository context.
2. Spin up the repo in a remote container.
3. Draft an instruction and a private verifier.
4. Package everything as a Harbor task.
5. Iterate through a review‑and‑repair loop until the task passes all controls.

Where to find more

• Docs & guides – https://huggingface.github.io/Repo2RLEnv/
• Datasets – https://huggingface.co/collections/FineEnvs/repo2rlenv-verifiable-rl-environments-6aa82300d7494c050f50508d
• Source code – https://github.com/huggingface/Repo2RLEnv
• License – Apache‑2.0 for the library, MIT for bundled research recipes (distribution: Apache-2.0 AND MIT).

Who should use it

• Researchers building or evaluating code‑generation, bug‑fix, or software‑engineering agents.
• Teams that want to create custom RL benchmarks from their own codebases.
• Anyone interested in large‑scale, automatically verified programming environments.