allenai/ScienceWorld

ScienceWorld is a text-based virtual environment centered around accomplishing tasks from the standardized elementary science curriculum.

What it solves

ScienceWorld provides a standardized way to test if AI agents can perform complex, multi-step tasks based on elementary science curriculum. It addresses the need for a virtual environment where textual agents can be evaluated on their ability to reason and interact with a world governed by basic scientific laws.

How it works

It is a text-based virtual environment written in Scala and Java, accessible via a Python API. The system simulates various scientific scenarios—such as boiling water, testing electrical conductivity, or growing plants—and requires agents to interact with the environment to achieve specific goals. It includes a set of predefined tasks with numerous variations to prevent agents from simply memorizing solutions.

Who it’s for

Researchers and developers building textual AI agents who want to benchmark their agent's reasoning and problem-solving capabilities in a scientific context.

Highlights

  • Curriculum-based tasks: Covers a wide range of elementary science topics including chemistry, biology, and physics.
  • Extensive variations: Offers thousands of task variations across different categories to ensure robust testing.
  • Simplification presets: Includes options to lower the difficulty (e.g., teleporting or auto-opening doors) to help agents learn more easily.
  • Multi-interface access: Supports a Python API for programmatic control, a command-line interface, and a web server demo for human interaction.

Related

  • Project
    xlang-ai/OSWorld

    A benchmark and sandboxed environment for evaluating multimodal AI agents on their ability to complete open-ended tasks in real computer operating systems.

  • Project
    google-research/android_world

    An environment and benchmark for building and testing autonomous agents that can control Android devices across a wide variety of real-world apps.

  • Project
    QwenLM/Qwen-AgentWorld

    A native language world model that simulates agentic environments across seven domains to provide scalable and controllable training and evaluation for AI agents.

  • Project
    benchflow-ai/benchflow

    A universal environment framework that runs AI agents against task environments and scores them through a standardized contract, treating benchmarks as frozen environments.

  • Project
    ServiceNow/BrowserGym

    An open-source framework that provides a unified interface for researching and evaluating web agents across multiple benchmarks and open-ended web tasks.