OpenAI PaperBench: Evaluating AI Agents' Ability to Replicate AI Research

TL;DR

OpenAI has introduced PaperBench, a benchmark that evaluates whether AI agents can autonomously replicate state-of-the-art AI research. This matters because it tests the high-level AI engineering capabilities of agents, including the ability to understand complex research papers, develop codebases from scratch, and execute experiments successfully.

PaperBench Benchmark Design

PaperBench evaluates AI agents on their ability to replicate 20 ICML 2024 Spotlight and Oral papers. To successfully replicate a paper, an agent must be able to understand the paper's contributions, develop a complete codebase, and execute experiments to reproduce the results.

To ensure objective evaluation, OpenAI developed rubrics that decompose each replication task into 8,316 individually gradable sub-tasks. These rubrics were co-developed with the original authors of the ICML papers to ensure accuracy and realism in the grading criteria.

Evaluation Framework and LLM-based Judging

To enable scalable evaluation, OpenAI implemented an LLM-based judge to automatically grade replication attempts against the established rubrics. To verify the accuracy of this automated judge, the team created a separate benchmark specifically to assess the judge's performance.

Performance Results

OpenAI evaluated several frontier models on PaperBench. The best-performing agent tested—Claude 3.5 Sonnet (New) using open-source scaffolding—achieved an average replication score of 21.0%.

Comparing AI agents to human baselines, OpenAI recruited top ML PhDs to attempt a subset of the benchmark. The results indicate that current frontier models do not yet outperform the human baseline in replicating AI research.

Availability and Research Goals

OpenAI has open-sourced the PaperBench code to facilitate further research into the AI engineering capabilities of AI agents, making the the codebase available on GitHub via the preparedness repository.