OpenAI Sharing AI Progress in Mathematics

OpenAI has released a a set of mathematical results generated by an internal frontier model to advance the frontier of human knowledge. This release is designed to facilitate scientific transparency and allow the mathematics community to verify and build upon AI-generated proofs.

Verification via Lean Formalizations

OpenAI is providing formalizations of many of the proofs in Lean, a programming language that enables computer-checked mathematical proofs. The company has committed to updating the GitHub repository containing these results with additional formalizations as they become available to ensure the highest level of verification.

Transparency and Methodology

To ensure scientific openness, OpenAI has published specific details regarding the process used to obtain the results in their GitHub repository. The published transparency data includes:

  • Reasoning Summaries: 10 summaries detailing the model's reasoning process.
  • Compute Estimations: Estimates of the compute spent, measured in terms of ChatGPT Pro usage.
  • Attempt Statistics: Statistics regarding the number of problems attempted.

According to OpenAI, the average result was produced using compute equivalent to approximately three hours of ChatGPT Pro "thinking."

Community Engagement and Standards

OpenAI has been consulting with the Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study (AGMAI) to develop best practices for sharing AI-generated mathematical results. The current release follows AGMAI's public recommendations and utilizes a GitHub repository with established protocols for paper revisions and citations.

OpenAI has stated its commitment to improving the quality of mathematical exposition and presentation in future releases to enhance community understanding. Additionally, the company plans to fund workshops, conferences, and special programs focused on the understanding of major AI-produced mathematical results.

Model Availability

While the results are currently shared as papers and formalizations, OpenAI is working toward the responsible release of the internal frontier model that produced these findings. The company emphasizes the ongoing evaluation of internal frontier models in mathematics and other sciences to accelerate the development of scientific tools.

Sources