InternLM/WildClawBench

What it solves

WildClawBench is designed to move beyond isolated capability tests (like simple function calling or JSON parsing) to evaluate whether AI agents can perform complex, end-to-end real-world work. It addresses the gap between benchmark performance and actual utility by testing agents in live environments with high-stakes, long-horizon tasks that require autonomous planning and error recovery.

How it works

Agents are deployed into a live OpenClaw environment—a personal AI assistant framework—with access to real tools such as a web browser, bash terminal, file system, email, and calendar. The benchmark consists of 60 original, hand-built tasks across six categories: Productivity Flow, Code Intelligence, Social Interaction, Search & Retrieval, Creative Synthesis, and Safety Alignment.

To ensure reproducibility and prevent data leakage, each task runs in an isolated Docker container. The system supports four different agent harnesses (OpenClaw, Claude Code, Codex CLI, and Hermes Agent), allowing researchers to distinguish between the capabilities of the underlying LLM and the scaffolding provided by the agent framework.

Who it’s for

This tool is for AI researchers and developers building agentic systems who need a rigorous, practical evaluation suite to measure the autonomy, multimodal capabilities, and safety of their models in real-world scenarios.

Highlights

• Real-world environments: Uses live tools instead of mocks to test actual agency.
• Diverse task suite: 60 original tasks covering everything from clipping football highlights to negotiating emails and debugging undocumented code.
• Harness comparison: Supports four different agent frameworks to isolate model performance from scaffolding.
• Isolated execution: Each task runs in its own Docker container for reproducibility and security.
• Multimodal testing: Includes tasks requiring video understanding and cross-modal synthesis.