OpenAI GDPval: Measuring Model Performance on Real-World Knowledge Work

OpenAI has introduced GDPval, a new evaluation framework designed to track how AI models perform on economically valuable, real-world tasks. By drawing tasks from the industries and occupations that contribute most to the U.S. Gross Domestic Product (GDP), GDPval shifts AI evaluation from academic benchmarks toward measuring tangible professional productivity.

GDPval Framework and Methodology

GDPval measures model performance across 44 occupations spanning the top 9 industries that contribute over 5% to the U.S. GDP. The evaluation consists of 1,320 specialized tasks, with a "gold" open-sourced set of 220 tasks.

Task Selection and Design

To ensure economic relevance, OpenAI used a specific selection process for occupations:

  • Industry Selection: Based on Federal Reserve Bank of St. Louis data, the top 9 industries contributing over 5% to U.S. GDP were selected.
  • Occupation Selection: Within those industries, the 5 occupations contributing most to total wages and compensation were chosen, provided they were predominantly knowledge work.
  • Knowledge Work Threshold: Using O*NET data, an occupation qualified if at least 60% of its component tasks did not involve physical work or manual labor.

Dataset Construction

Tasks were crafted by professionals averaging 14 years of experience. Each task is based on actual work products—such as nursing care plans, engineering blueprints, or legal briefs—rather than synthetic exam questions. Each task underwent an average of five rounds of expert review, including model-based validation and peer checks, to ensure they were representative and feasible.

Unlike traditional benchmarks, GDPval tasks include reference files and context, and require deliverables that may include spreadsheets, diagrams, slides, or multimedia documents.

Model Performance and Grading

Model performance is evaluated through blind comparisons conducted by expert "graders" from the same professional fields. These graders rank AI-generated deliverables against human-produced work, classifying them as "better," "as good as," or "worse than" the human expert.

Key Findings and Benchmarks

Blind evaluations involving GPT-4o, o4-mini, OpenAI o3, GPT-5, Claude Opus 4.1, Gemini 2.5 Pro, and Grok 4 revealed the following:

  • Human Parity: Frontier models are approaching the quality of industry experts. Claude Opus 4.1 was rated as good as or better than humans in nearly half of the tasks, showing particular strength in aesthetics and layout.
  • Accuracy: GPT-5 demonstrated particular excellence in accuracy and domain-specific knowledge.
  • Growth Trend: Performance has more than doubled from GPT-4o (Spring 2024) to GPT-5 (Summer 2025), following a linear trend.
  • Efficiency: Frontier models can complete these tasks approximately 100x faster and 100x cheaper than human experts, though these figures only account for inference and API costs, not human oversight.

Automated Grading

OpenAI has developed an "automated grader," an AI system trained to predict human expert judgments. While available as an experimental research service at evals.openai.com, it is not yet used to replace human expert graders due to reliability gaps.

Limitations and Future Development

GDPval is currently a one-shot evaluation, meaning it does not capture the iterative nature of professional work, such as revising a document based on client feedback. It also does not account for the ambiguity inherent in real-world tasks where a professional must first determine the correct approach before executing a deliverable.

Future iterations of GDPval will aim to:

  • Expand the number of covered occupations and industries.
  • Incorporate interactive workflows and multi-draft iterations.
  • Include tasks that require navigating ambiguity to better reflect the complexity of professional knowledge work.

Sources