openenv/OpenEnv

What it solves

Training agents with reinforcement learning (RL) is often slowed down by the lack of a standardized way to create, deploy, and interact with isolated environments. Every task (a game, a coding sandbox, a browser) comes with its own interface, making it hard to compare results or reuse code. OpenEnv solves this by providing a single, simple API to connect to many different environments, each packaged as a lightweight Docker image, so researchers can focus on training instead of plumbing.

How it works

OpenEnv uses a Gymnasium-style API with three core methods: reset(), step(), and state(), available both synchronously and asynchronously over a WebSocket. Each environment is a Docker image that can run locally, on a cloud sandbox, or on Hugging Face Spaces. The package includes command-line tools (openenv init, validate, push) to scaffold, check, and deploy your own environments. For training, OpenEnv integrates with popular RL frameworks (TRL, Unsloth, SkyRL, and more) either as white-box tool environments (the trainer runs the loop) or as loop-owning harnesses (the agent runs its own loop while OpenEnv captures training traces). It also exposes environment tools via MCP in production mode, and offers built-in reward composing and evaluation through Inspect AI.

Who it’s for

OpenEnv is aimed at RL researchers and practitioners who want to train agents on diverse tasks — games, coding, web browsing, simulations — without reinventing the environment plumbing. It’s also for environment builders who want a quick path to package and share their environments with the community.

Highlights

• One API for every environment: reset(), step(), state() over WebSocket, sync or async.
• 40+ ready-to-use environments (Echo, Coding, TextArena, OpenSpiel, BrowserGym, Harbor, and more).
• Integrates with TRL, Unsloth, SkyRL, ART, Oumi, torchforge, Miles, and other RL frameworks.
• Real coding-agent training via Harbor, capturing exact tokens for reinforcement learning.
• MCP support to serve environment tools to agents in production, while training keeps using step() and rewards.
• Built-in reward composition with rubrics and evaluation with Inspect AI.