ServiceNow/BrowserGym

What it solves

BrowserGym provides a unified, open-source framework to accelerate the research and development of web agents. It simplifies the process of testing and evaluating AI agents that interact with web browsers, removing the need to create separate setups for various different web-based benchmarks.

How it works

BrowserGym integrates multiple web agent benchmarks into a single ecosystem using the Gymnasium interface. By wrapping these benchmarks as Gymnasium environments, it allows researchers to create a loop where an agent can take actions in a browser (via Playwright) and receive observations. It also provides an open-ended task mode for interactive, real-world web navigation.

Who it’s for

It is designed for AI researchers and developers building web agents, specifically those who need a standardized way to implement, test, and evaluate their agents across diverse web environments.

Highlights

• Unified Benchmark Access: Integrates a wide array of benchmarks including MiniWoB, WebArena, VisualWebArena, WorkArena, AssistantBench, and others.
• Extensible Design: New benchmarks can be easily added by inheriting from the AbstractBrowserTask class.
• Gymnasium Integration: Uses the standard gymnasium API for environment interaction, making it compatible with existing RL and agentic workflows.
• Open-Ended Tasks: Supports interactive chat-based tasks on the open web starting from any URL.
• Demo Agent: Includes a demo agent powered by LLMs (like GPT-4o-mini) to get users started quickly.