GPT-4o mini release notes / what's new

OpenAI has introduced GPT-4o mini, a cost-efficient small model designed to make high-performance AI more affordable and accessible. The model is positioned as a replacement for GPT-3.5 Turbo, offering superior textual intelligence and multimodal reasoning at a significantly lower price point.

Technical Specifications and Capabilities

GPT-4o mini is a multimodal model supporting text and vision in the API, with future support planned for image, video, and audio inputs and outputs. It features a 128K token context window and supports up to 16K output tokens per request. The model's knowledge cutoff is October 2023.

Key technical advantages include:

  • Improved Tokenizer: Shared with GPT-4o, this tokenizer makes handling non-English text more cost-effective.
  • Long-Context Performance: The model demonstrates improved performance in long-context scenarios compared to GPT-3.5 Turbo.
  • Function Calling: Strong performance in function calling allows developers to build applications that interact with external systems to fetch data or take actions.

Performance Benchmarks

GPT-4o mini outperforms GPT-3.5 Turbo and other small models across several academic benchmarks:

Reasoning and Textual Intelligence

GPT-4o mini scored 82.0% on MMLU, surpassing Gemini Flash (77.9%) and Claude Haiku (73.8%).

Mathematics and Coding

  • MGSM (Math Reasoning): GPT-4o mini scored 87.0%, compared to 75.5% for Gemini Flash and 71.7% for Claude Haiku.
  • HumanEval (Coding): GPT-4o mini scored 87.2%, compared to 71.5% for Gemini Flash and 75.9% for Claude Haiku.

Multimodal Reasoning

On the MMMU benchmark, GPT-4o mini scored 59.4%, outperforming Gemini Flash (56.1%) and Claude Haiku (50.2%).

Cost and Availability

GPT-4o mini is an order of magnitude more affordable than previous frontier models and over 60% cheaper than GPT-3.5 Turbo. Pricing is set at 15 cents per million input tokens and 60 cents per million output tokens.

Availability:

  • API: Now available in the Assistants API, Chat Completions API, and Batch API.
  • ChatGPT: Available for Free, Plus, and Team users immediately; Enterprise users will gain access starting next week.
  • Fine-tuning: Planned for rollout in the coming days.

Safety and Security

Safety is integrated into the development process through pre-training filters (removing hate speech, adult content, and spam) and post-training alignment using reinforcement learning with human feedback (RLHF).

GPT-4o mini is the first model to apply the instruction hierarchy method, which enhances the model's ability to resist jailbreaks, prompt injections, and system prompt extractions, making it more reliable for large-scale applications.

Use Cases and Industry Application

Low cost and low latency make GPT-4o mini suitable for applications that chain or parallelize multiple model calls, pass large volumes of context (such as full code bases), or provide real-time customer support chatbots. Partners like Ramp and Superhuman have found the model to perform significantly better than GPT-3.5 Turbo for extracting structured data from receipts or generating high-quality email responses based on thread history.

Sources