huggingface/OpenEnv

An interface library for RL post training with environments.

OpenEnv – Agentic Execution Environments

What it is – OpenEnv is a Python library that lets you run isolated reinforcement‑learning environments (each packaged as a Docker image) and interact with them through a simple Gymnasium‑style API (reset(), step(), state()). Environments can be hosted locally, in a cloud sandbox, or as a Hugging Face Space, and they communicate over WebSockets, supporting both async and synchronous usage.

Why it matters – Modern RL for LLM‑powered agents needs safe, reproducible sandboxes where the agent can call tools, receive observations, and get rewards. OpenEnv provides a unified way to create, share, and train on such sandboxes, with built‑in reward composition, evaluation utilities, and tight integration with popular RL frameworks (TRL, ART, SkyRL, etc.).

Key features

  • Single API – reset(), step(), state() work the same for every environment, async or sync.
  • Docker‑isolated environments – each env runs in its own container, making it safe to execute code, run browsers, or simulate finance tasks.
  • Catalog of 40+ ready‑to‑use envs – games (Wordle, OpenSpiel), coding sandboxes, browser navigation (BrowserGym), finance simulators, and the Harbor harness for coding‑agent datasets.
  • Framework‑agnostic training – works with TRL, Unsloth, SkyRL, ART, Miles, Oumi, torchforge, etc.; examples include GRPO trainers and async harness training.
  • Tool‑exposing environments – MCP environments expose tool endpoints (/mcp) for agents in production while still using the standard step() loop for training.
  • Reward & evaluation helpers – compose rewards with rubrics and evaluate with the Inspect AI suite.
  • CLI & SDK – openenv init, validate, push, and import commands make it easy to scaffold, test, and deploy new environments.

Typical workflow

  1. Pick or build an environment – use one from the catalog (e.g., Echo, Coding, BrowserGym) or run openenv init my_env to scaffold a new one.
  2. Deploy – openenv push sends the Docker image to a Hugging Face Space or cloud sandbox.
  3. Connect from code – install the client package (pip install git+https://huggingface.co/spaces/openenv/echo_env) and use the async or sync context manager to call reset() and step().
  4. Train – pass an environment_factory to a TRL trainer (e.g., GRPOTrainer) or use the Harbor integration to capture every model call as a TrainingTrace for RL‑from‑human‑feedback style training.
  5. Evaluate – compose reward rubrics and run Inspect AI evaluations.

Getting started (quick)

pip install openenv                     # core SDK
pip install git+https://huggingface.co/spaces/openenv/echo_env  # example client

import asyncio
from echo_env import CallToolAction, EchoEnv

async def main():
    async with EchoEnv(base_url="https://openenv-echo-env.hf.space") as env:
        await env.reset()
        res = await env.step(CallToolAction(tool_name="echo_message",
                                            arguments={"message": "Hello"}))
        print(res.observation.result["data"])  # → Hello
        print(res.reward)

asyncio.run(main())

For synchronous code, wrap the client with .sync().

Documentation & community – Full docs, tutorials, and a Discord are linked in the README. The project is under a BSD‑3‑Clause license and is early‑stage, so APIs may evolve.

All information above is taken directly from the repository’s README; no external assumptions have been added.

Related