huggingface/Repo2RLEnv

Convert any Repo into an RL Environment

What it solves

Repo2RLEnv converts existing GitHub repositories into verifiable Reinforcement Learning (RL) environments. This allows developers to create high-quality training and evaluation datasets for coding agents without needing to manually write tasks or human-graded rewards, as the environments provide programmatic, automated scoring.

How it works

The tool uses synthesis pipelines to analyze a repository's source code, merged pull requests (PRs), and commits to generate tasks. These tasks are exported in the Harbor spec, making them compatible with various RL runtimes and agent harnesses.

Key processes include:

  • Synthesis Pipelines: Different strategies to create tasks, such as mining PR diffs (pr_diff) or running existing test suites in Docker sandboxes (pr_runtime and commit_runtime).
  • Bootstrap Phase: For runtime-based pipelines, an LLM agent automatically configures a Docker environment where the repository's tests can run, which is then cached for efficiency.
  • Verification: Rewards are calculated based on either the execution of the repository's own tests (test execution) or by comparing the agent's output to a known correct diff (diff similarity).

Who it’s for

It is designed for researchers and developers training or evaluating LLM-based coding agents who need scalable, verifiable data from real-world software projects.

Highlights

  • Verifiable Rewards: Uses programmatic signals (test passes/fails or diff similarity) instead of human grading.
  • Harbor Compatibility: Outputs data in the Harbor spec for seamless integration with RL trainers and agent harnesses.
  • Automated Environment Setup: The bootstrap phase uses an LLM to automatically build the necessary Docker sandboxes.
  • Multi-Language Support: Supports Python, Go, Node, and Rust for runtime pipelines, and any language for text-only diff pipelines.
  • Hub Integration: Natively pushes and pulls datasets directly to and from the Hugging Face Hub.

Related

  • Project
    harbor-framework/harbor

    A framework for evaluating and optimizing AI agents and language models by running benchmarks across scalable, containerized environments.

  • Project
    rllm-org/rllm

    An open-source framework for training language agents with reinforcement learning, allowing the same agent code to be used across different backends, sandboxes, and benchmarks.

  • Project
    InternLM/InternBootcamp

    A scalable synthetic environment framework for training and evaluating LLM agents through executable, interactive, and verifiable tasks.

  • Project
    huggingface/OpenEnv

    An end-to-end framework for creating and deploying isolated, sandboxed execution environments for agentic RL training using a standardized Gymnasium-style API.

  • Project
    Snowflake-Labs/agent-world-model

    A synthetic environment generation pipeline that creates thousands of SQL-backed tool-use environments for large-scale agentic reinforcement learning.