NVIDIA-NeMo/Gym
Evaluate and improve models and agents using environments
What it solves
NeMo Gym is designed to evaluate and improve AI models and agents, particularly those operating in stateful environments like code execution, tool calling, and sandboxes. It provides a scalable infrastructure to run evaluations and training runs across thousands of concurrent requests, ensuring reproducibility across teams using shared environments and verifiers.
How it works
The library uses a modular system where an "environment" consists of a dataset of tasks, an agent harness (how the model interacts with the world), and a verifier (how the task completion is scored). It integrates with various inference providers (such as vLLM, OpenAI, and Together.ai) and training frameworks (like NeMo RL, Unsloth, and VeRL) to allow a seamless transition from evaluation to agent optimization and training.
Who it’s for
It is intended for developers and researchers working on AI agents and LLMs who need to perform large-scale, reproducible evaluations or use environments for Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) training.
Highlights
- Scalable Evaluation: Support for thousands of concurrent environments for high-throughput evaluation and training.
- Environment Hub: A collection of popular benchmarks and training environments across domains like coding, knowledge, and science.
- Pluggable Sandboxes: Ability to run tool-using agents in isolated environments via various sandbox providers.
- Modular Architecture: Extensible interfaces for agents, tasks, and verifiers, allowing users to bring their own agents or use built-in harnesses.
- Integrated Ecosystem: Part of the NVIDIA NeMo platform, integrating with GPU-accelerated training and optimization tools.