InternLM/WildClawBench

What it solves

WildClawBench is designed to move beyond isolated capability tests (like simple function calling or JSON parsing) to evaluate whether AI agents can perform complex, end-to-end real-world work. It addresses the gap between benchmark scores and actual utility by testing agents in live environments with long-horizon tasks that require autonomous planning, error recovery, and multi-modal synthesis.

How it works

Agents are deployed into a live OpenClaw environment—a personal AI assistant framework—equipped with real tools such as a web browser, bash terminal, file system, email, and calendar. The benchmark consists of 60 original, hand-built tasks across six categories (Productivity, Code Intelligence, Social Interaction, Search & Retrieval, Creative Synthesis, and Safety Alignment).

To ensure reproducibility and prevent data leakage, each task runs in an isolated Docker container. The system supports four different agent harnesses (OpenClaw, Claude Code, Codex CLI, and Hermes Agent), allowing researchers to distinguish between the capabilities of the underlying LLM and the scaffolding provided by the agent framework.

Who it’s for

This tool is for AI researchers and developers building agentic systems who need a rigorous, practical way to measure the performance of their models in complex, multi-step workflows.

Highlights

• Real-world environments: Uses live tools instead of mocks to simulate actual user workflows.
• Long-horizon tasks: Tests workflows spanning 10-20 minutes of execution with 10-60+ tool calls.
• Multi-modal evaluation: Includes tasks requiring video understanding, image generation, and cross-modal synthesis.
• Harness comparison: Supports multiple agent scaffolds to isolate model performance from framework efficiency.
• Isolated execution: Uses Docker containers for each task to ensure consistent and reproducible results.