NVIDIA-NeMo/Gym

🎯 What NeMo Gym is

NeMo Gym (the Gym component of NVIDIA’s NeMo platform) is a Python library that lets you evaluate and train AI agents inside stateful environments. An environment bundles a dataset, a sandbox where the model can run code or call tools, a verifier that scores the result, and the per‑task state. The library provides:

Modular interfaces for agents, environments, tasks, and verifiers.
An “environment hub” with dozens of ready‑made benchmarks (multiple‑choice QA, code‑generation, safety, reasoning, etc.).
Scalable execution – run thousands of concurrent environments, repeat tasks, or launch jobs across GPUs/Slurm nodes.
Agent harnesses that wrap models (OpenAI, vLLM, Fireworks, etc.) so they can interact with the sandbox.
Training support – plug the environments into any RL framework (NeMo RL, VeRL, Unsloth) for reinforcement‑learning or fine‑tuning.
Optional OpenTelemetry tracing for end‑to‑end observability.

It is designed for teams that need reproducible, large‑scale evaluation of agentic or tool‑calling models, and for researchers who want to improve those agents with RL or supervised fine‑tuning.

🚀 Quick‑Start (from the README)

# Clone and set up a virtual environment (requires uv & Python 3.13+)
git clone https://github.com/NVIDIA-NeMo/Gym.git
cd Gym
uv venv --python 3.13 && source .venv/bin/activate
uv sync

Create a simple env.yaml pointing at an OpenAI model (any other provider works similarly):

policy_base_url: https://api.openai.com/v1
policy_api_key: <your‑openai‑api‑key>
policy_model_name: gpt-4.1-2025-04-14

Start the three local servers that coordinate the dataset, the agent, and the model:

gym env start \
    --resources-server mcqa \
    --model-type openai_model

Run a tiny evaluation (5 example MCQA tasks, single repeat):

gym eval run --no-serve \
    --agent mcqa_simple_agent \
    --input resources_servers/mcqa/data/example.jsonl \
    --output results/mcqa_rollouts.jsonl \
    --limit 5 \
    --num-repeats 1

The CLI prints aggregate metrics, e.g.:

{
  "mean/reward": 0.8,
  "pass@1[avg-of-1]/accuracy": 80.0,
  "pass@1/accuracy": 80.0
}

📦 Core Features (as listed in the README)

Feature	What it means
Modular, extensible interfaces	Plug‑in your own agent code, environment logic, or verifier without changing the core library.
Environment hub	Hundreds of pre‑built benchmarks across domains (code, safety, reasoning, multilingual, long‑context, etc.).
Scalable execution	Run many environments in parallel, schedule jobs on GPUs or Slurm, and automatically resume after pre‑emptions.
RL‑framework agnostic	Use NeMo RL, VeRL, Unsloth, or any other RL trainer with the same environment objects.
OpenTelemetry tracing	Optional end‑to‑end observability of model calls, tool usage, and latency.
Production‑tested	Used internally for training NVIDIA’s Nemotron series.

🛠️ Typical Use‑Cases

Scenario	How NeMo Gym helps
Benchmarking a new LLM	Run the same environment (e.g., MCQA, code‑generation) across many repeats, collect rollouts, and get reproducible metrics.
Developing a tool‑calling agent	Use sandbox providers (Docker, ECS Fargate, OpenSandbox, etc.) to safely execute code or API calls, with automatic cleanup and failure handling.
RL fine‑tuning	Feed rollouts from the environment into an RL algorithm (e.g., PPO, GDPO) to improve the policy on a specific task distribution.
Safety / content‑moderation evaluation	Run multimodal safety environments (Aegis v4) that score prompts and responses with a content‑safety model.
Custom research environments	Follow the “environment tutorials” to build a new dataset‑driven task (single‑step, multi‑step, stateful) and plug it into the same training pipeline.

📚 Documentation & Community

Official docs – https://docs.nvidia.com/nemo/gym/main/ (installation, tutorials, CLI reference).
GitHub CI – unit‑test badge shows continuous integration.
Community – NVIDIA NeMo forums and Discord channels (linked from the docs).
Citations – The README includes a “Citations” section for academic use.

📦 Installation Summary

Python ≥ 3.13.14 (the README mentions 3.13.14 or higher).
No GPU required for the library itself; only needed for model inference in some environments.
Dependencies are installed via uv sync; Ray is pulled in automatically.
Optional: For VLM or audio/video benchmarks, install excluded codec packages via docker/install_excluded_deps.sh.

🎉 Bottom Line

NeMo Gym is a real, production‑grade framework for building, evaluating, and training AI agents that interact with complex, stateful environments. It sits inside NVIDIA’s broader NeMo ecosystem and is geared toward researchers and engineers who need scalable, reproducible benchmarking and RL‑style training for tool‑using or code‑generating models.