StanfordVL/BEHAVIOR-1K

BEHAVIOR-1K: a platform for accelerating Embodied AI research. Join our Discord for support: https://discord.gg/bccR5vGFEx

What it solves

BEHAVIOR-1K addresses the lack of comprehensive, human-centered benchmarks for embodied AI. It provides a standardized way to test how well AI agents can perform a wide variety of everyday household activities in a simulated environment.

How it works

The project provides a monolithic repository containing the simulation environment and tools necessary to train and evaluate agents. The benchmark consists of 1,000 distinct household activities—such as cleaning, cooking, and organizing—which were selected based on real human time-use surveys and preference studies to ensure they reflect actual human needs.

Who it’s for

This is designed for researchers and developers working on embodied AI, robotics, and agent training who need a realistic simulation benchmark to evaluate their agents' capabilities across a diverse set of tasks.

Highlights

  • 1,000 everyday household activities for testing agents.
  • Based on real-world human time-use surveys and preference studies.
  • Comprehensive simulation benchmark for training and evaluation.
  • Modular installation process for dependencies and components.

Related

  • Project
    facebookresearch/habitat-lab

    A modular high-level library for end-to-end development in embodied AI, used to train and evaluate agents performing tasks in indoor environments.

  • Project
    xavierpuigf/virtualhome

    An interactive simulation platform for household activities that allows researchers to train embodied AI agents and generate synthetic video data via a Python API.

  • Project
    benchflow-ai/skillsbench

    A benchmark for evaluating how AI agents leverage and compose modular skills—instructions, scripts, and resources—to perform specialized workflows.

  • Project
    TIGER-AI-Lab/ClawBench

    ClawBench is an open‑source benchmark that evaluates AI browser agents on 281 real‑world web tasks (e.g., booking travel, ordering food). It provides task specs, a multi‑layer trace recorder, and an LLM judge (deepseek‑v4‑pro) to score success. Install via `pip install clawbench-eval`, configure your model and judge API key, then run tasks in a Docker/Podman‑based Chromium harness. Results are posted to a public leaderboard and the full data are available on Hugging Face.

  • Project
    harbor-framework/terminal-bench

    Terminal-Bench is a continuous benchmark for measuring the capabilities of frontier AI agents using a diverse and evolving set of difficult tasks.