NVIDIA-NeMo/Gym

What it solves

NeMo Gym is a library designed to evaluate and improve AI models and agents, particularly those operating in stateful environments. It addresses the challenge of running reproducible, scalable evaluations and training runs across teams, especially for complex tasks like code execution, tool calling, and sandboxed environments where a simple stateless check of model outputs is insufficient.

How it works

The library provides a modular infrastructure consisting of four key components: a dataset (tasks to solve), an agent harness (how the model interacts with the world), a verifier (scoring task completion), and state (the execution context for each task). It uses local servers to coordinate between the model, the agent, and the task verification process. It can scale to thousands of concurrent environments and integrates with various training frameworks (like NeMo RL, Unsloth, and VeRL) for SFT and RL training.

Who it’s for

It is intended for developers and researchers working with AI agents and LLMs who need to perform large-scale, reproducible evaluation and evaluation-driven training (RLHF/RL) in complex, stateful environments.

Highlights

• Scalable Evaluation: Supports thousands of concurrent environments and multiple repeats per task.
• Modular Architecture: Extensible interfaces for agents, tasks, and verifiers.
• Environment Hub: Includes a collection of popular benchmarks and training environments across domains like coding, reasoning, and instruction following.
• Training Integration: Seamlessly transitions from evaluation to agent optimization and RL training.
• Observability: Optional OpenTelemetry tracing across agent, model, and resource servers.
• Pluggable Sandboxes: Supports multiple sandbox providers including Docker, Daytona, ECS Fargate, and others.