OpenAI GPT-4 announcement and capabilities overview

TL;DR

OpenAI released GPT‑4, a multimodal (text‑and‑image) large language model that reaches top‑10% scores on a simulated bar exam and shows human‑level performance across a wide range of professional and academic benchmarks; the model is being rolled out through ChatGPT, a gated API, and an open‑source evaluation framework.

Core Technical Advances

Predictable scaling and stable training

  • Over the past two years OpenAI rebuilt its deep‑learning stack and co‑designed a supercomputer with Azure specifically for its workloads.
  • GPT‑3.5 served as a test run; bugs were fixed and theoretical foundations improved before training GPT‑4.
  • GPT‑4’s training run was unusually stable, allowing OpenAI to accurately predict final loss and even extrapolate pass rates on downstream tasks from runs using 1,000‑10,000× less compute.
  • Predictable scaling is framed as a safety measure, enabling the organization to anticipate future model capabilities.

Multimodal input capability

  • GPT‑4 accepts interleaved text and image inputs and generates text outputs (including code) for vision‑language tasks.
  • Performance on standard academic vision benchmarks is strong, though the image modality remains a research preview and is not yet publicly available.
  • Test‑time techniques such as few‑shot prompting and chain‑of‑thought reasoning, originally developed for text‑only models, also improve image‑input tasks.

Alignment and steerability improvements

  • Six months of iterative alignment, informed by adversarial testing and ChatGPT experience, yielded the best‑ever factuality, steerability, and guard‑rail adherence for OpenAI models.
  • System messages let developers (and soon ChatGPT users) specify tone, verbosity, and style, offering customizable behavior within policy bounds.
  • OpenAI acknowledges that system messages are currently the easiest jailbreak vector, and it encourages user feedback to harden these controls.

Benchmark Performance

Human‑level exam results

  • GPT‑4 scores in the top 10 % of test‑takers on a simulated bar exam, compared to GPT‑3.5’s bottom‑10 % placement.
  • No task‑specific training was performed; the model was evaluated on publicly available exams and recent practice tests.

Standard language model benchmarks

  • GPT‑4 outperforms existing large language models and most state‑of‑the‑art (SOTA) systems on traditional ML benchmarks.
  • On the MMLU suite (14,000 multiple‑choice items across 57 subjects), GPT‑4 surpasses GPT‑3.5 and other LLMs (Chinchilla, PaLM) in 24 of 26 tested languages, including low‑resource languages such as Latvian, Welsh, and Swahili.

Factuality and safety metrics

  • Internal adversarial factuality evaluations show a 40 % reduction in hallucinations relative to GPT‑3.5.
  • After RLHF post‑training, GPT‑4 demonstrates a large gap over GPT‑3.5 on TruthfulQA, resisting common misconceptions while still occasionally missing subtle details.

Limitations

  • GPT‑4 still hallucinates facts and makes reasoning errors; it is less reliable than humans in many real‑world scenarios.
  • Knowledge is cut off around September 2021; the model does not learn from new experiences.
  • Calibration of confidence scores degrades after RLHF, even though the base model is well‑calibrated.
  • Biases persist in outputs, and the model can be coaxed into disallowed behavior via jailbreak techniques.

Safety Risks and Mitigations

  • Risks mirror prior models (harmful advice, buggy code, misinformation) but are amplified by higher capability.
  • Over 50 external experts from AI alignment, cybersecurity, biorisk, and international security performed adversarial testing, informing data collection to improve refusal behavior for dangerous requests.
  • An additional safety reward signal during RLHF trains a GPT‑4 zero‑shot classifier to penalize disallowed content; this reduced responses to prohibited requests by 82 % compared to GPT‑3.5 and increased compliance on sensitive topics by 29 %.
  • OpenAI continues to rely on deployment‑time safety measures (monitoring, abuse detection) alongside model‑level interventions.

Deployment Details

ChatGPT Plus

  • GPT‑4 is available to ChatGPT Plus subscribers on chatgpt.com with a usage cap that will be adjusted based on demand and system performance.

API Access

  • Text‑only GPT‑4 access is provided via the standard ChatCompletions API; developers must join a waitlist.
  • Pricing: $0.03 per 1k prompt tokens and $0.06 per 1k completion tokens for the 8,192‑token context model; $0.06/$0.12 for the 32,768‑token context variant (gpt‑4‑32k).
  • Rate limits: 40k tokens/minute and 200 requests/minute by default.
  • Researchers can apply for subsidized access through the Researcher Access Program.

OpenAI Evals Framework

  • OpenAI released the open‑source OpenAI Evals framework for creating and running automated benchmarks.
  • Evals is used internally to identify shortcomings and prevent regressions; external users (e.g., Stripe) have leveraged it to evaluate GPT‑powered tools.
  • The repository includes templates for common evaluation patterns, including model‑graded evals where GPT‑4 judges its own outputs.
  • Community contributions are encouraged to broaden the set of failure modes and difficult tasks captured by the framework.

Implications and Outlook

  • GPT‑4’s multimodal abilities and improved alignment mark a significant step toward more useful and safer AI assistants.
  • Predictable scaling and early safety‑focused training aim to provide better foresight into future model capabilities, a practice OpenAI argues is critical for societal safety.
  • Ongoing limitations—hallucinations, bias, outdated knowledge—highlight the need for human oversight, especially in high‑stakes applications.
  • The open‑source Evals ecosystem and collaborative safety testing signal a shift toward broader community involvement in AI evaluation and risk mitigation.

This article faithfully reflects OpenAI’s March 2023 GPT‑4 announcement without adding or altering any factual content.

Sources