OpenAI GPT-4o Release Notes
OpenAI has introduced GPT-4o ("omni"), a multimodal model designed for more natural human-computer interaction. By processing text, audio, image, and video inputs and generating text, audio, and image outputs through a single neural network, GPT-4o achieves response times similar to human conversation.
End-to-End Multimodal Architecture
GPT-4o replaces the previous multi-model pipeline used in Voice Mode with a single end-to-end neural network. Previously, Voice Mode relied on three separate models: one for audio-to-text transcription, one for text processing (GPT-3.5 or GPT-4), and one for text-to-audio conversion. This pipeline caused significant information loss, as the core intelligence model could not directly observe tone, background noise, or multiple speakers, nor could it output emotion, singing, or laughter.
By training a single model across text, vision, and audio, GPT-4o processes all inputs and outputs within the same network, eliminating the pipeline latency and information loss associated with separate models.
Performance and Latency
GPT-4o matches GPT-4 Turbo's performance on English text and code while providing significant improvements in non-English language processing, vision, and audio understanding.
Response Speed
GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds. This is a substantial reduction in latency compared to the previous Voice Mode, which averaged 2.8 seconds for GPT-3.5 and 5.4 seconds for GPT-4.
Multilingual Tokenization
GPT-4o utilizes a new tokenizer that significantly improves compression across various language families. Examples of token reduction include:
- Gujarati: 4.4x fewer tokens
- Telugu: 3.5x fewer tokens
- Tamil: 3.3x fewer tokens
- Marathi: 2.9x fewer tokens
- Hindi: 2.9x fewer tokens
- Arabic: 2.0x fewer tokens
- Russian: 1.7x fewer tokens
- Korean: 1.7x fewer tokens
- Chinese: 1.4x fewer tokens
- Japanese: 1.4x fewer tokens
- English: 1.1x fewer tokens
Safety and Risk Assessment
GPT-4o incorporates safety by design through training data filtering and post-training behavior refinement. OpenAI has implemented new safety systems specifically for voice outputs and conducted external red teaming with over 70 experts across domains such as misinformation, bias, and social psychology.
Risk Scorecard
According to OpenAI's Preparedness Framework, GPT-4o does not score above "Medium" risk in any category. The risk levels both pre- and post-mitigation are as follows:
| Category | Pre-mitigation Risk | Post-mitigation Risk |
|---|---|---|
| Cybersecurity | Low | Low |
| CBRN | Low | Low |
| Persuasion | Medium | Medium |
| Model Autonomy | Low | Low |
Availability and API Pricing
GPT-4o is available to both free and Plus users in ChatGPT, with Plus users receiving up to 5x higher message limits. For developers, GPT-4o is available in the API as a text and vision model with the following improvements over GPT-4 Turbo:
- Speed: 2x faster
- Cost: 50% cheaper (half the price)
- Rate Limits: 5x higher rate limits
Audio and video capabilities will be rolled out iteratively to trusted partners and ChatGPT Plus alpha users in the coming weeks.
Sources
- OriginalHello GPT-4o