OpenAI Solving Math Word Problems and the GSM8K Dataset

OpenAI has developed a system capable of solving grade school math word problems with nearly twice the accuracy of a fine-tuned GPT-3 model. The system achieved a 55% success rate on a new benchmark, performing at approximately 90% of the level of a small sample of 9-12 year old children, who scored 60% on the same test.

Overcoming Multistep Reasoning Failures

Large language models like GPT-3 typically struggle with tasks requiring accurate multistep reasoning. While these models can mimic the style and cadence of a correct mathematical solution, they frequently commit critical logical errors. Because autoregressive models generate tokens sequentially, they lack a native mechanism to correct errors once they occur; a single mistake often makes the rest of the solution unrecoverable.

To address this, OpenAI implemented a verification system that allows the model to recognize its own mistakes and iterate until a correct solution is found. This approach shifts the focus from simple generation to the evaluation of multiple proposed solutions.

The GSM8K Dataset

To evaluate and facilitate research into mathematical reasoning, OpenAI released the GSM8K dataset.

Dataset Characteristics

Composition: 8.5K high-quality grade school math word problems.
Complexity: Each problem requires between 2 and 8 steps to solve.
Operations: Solutions rely on basic arithmetic operations, including addition, subtraction, multiplication, and division.
Format: Solutions are written in natural language rather than pure mathematical expressions, making them more interpretable for humans and the method more domain-agnostic.

OpenAI noted that while the concepts in GSM8K are elementary, the high diversity of problems causes state-of-the-art fine-tuned language models to perform poorly.

Training Verifiers for Error Correction

OpenAI improved performance by training verifiers to evaluate whether a proposed solution is correct. Instead of relying on a single output, the system generates 100 candidate solutions for a given problem and uses the verifier to select the highest-ranked candidate.

Key Findings on Verification

Efficiency: Verification is often a simpler task than generation, providing a strong performance boost when the dataset is sufficiently large.
Scaling: On the full training set, a 6B parameter model using verification slightly outperformed a fine-tuned 175B parameter model. This suggests that verification provides a performance boost roughly equivalent to a 30x increase in model size.
Risk of Overfitting: With datasets that are too small, verifiers may overfit by memorizing final answers rather than learning the underlying properties of mathematical reasoning.

Implications for General AI

The ability to produce correct arguments and recognize incorrect ones is a fundamental requirement for general AI. Grade school math serves as an ideal testbed because the problems are conceptually simple, but highly sensitive to subtle errors. By training models to separate successful solutions from failed attempts, OpenAI aims to develop skills that will be applicable to more logically complex domains in the future.