OpenAI introduces GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper in the API

Overview

OpenAI announced three audio models for the Realtime API: GPT‑Realtime‑2, GPT‑Realtime‑Translate, and GPT‑Realtime‑Whisper. These models aim to let developers create voice experiences that feel more natural, respond more intelligently, and take action in real time.

GPT‑Realtime‑2 capabilities

GPT‑Realtime‑2 adds GPT‑5‑class reasoning to live voice interactions. Developers can select reasoning effort from minimal, low, medium, high, and xhigh, with low as the default. The model supports preambles such as “let me check that” to signal work, parallel tool calls with audible feedback, stronger recovery behavior, an expanded context window from 32K to 128K, better retention of domain‑specific terminology, and controllable tone and delivery.

On audio evaluations, GPT‑Realtime‑2 (high) scores 15.2% higher on Big Bench Audio for audio intelligence than GPT‑Realtime‑1.5, and GPT‑Realtime‑2 (xhigh) scores 13.8% higher on Audio MultiChallenge for instruction following. In a Zillow benchmark, prompt optimization yielded a 26‑point lift in call success rate (95% vs. 69%).

“What stood out about GPT-Realtime-2 was the intelligence and tool-calling reliability it brings to complex voice interactions. On our hardest adversarial benchmark, this translates to a 26-point lift in call success rate after prompt optimization (95% vs. 69%). GPT-Realtime-2 is also materially more robust on Fair Housing compliance, which is critical for our business. The combination of agentic competence and guardrail strength is what makes it viable for production voice at Zillow.” — Josh Weisberg, SVP and Head of AI at Zillow

GPT‑Realtime‑Translate capabilities

GPT‑Realtime‑Translate provides live speech translation from more than 70 input languages to 13 output languages while keeping pace with the speaker. It is intended for customer support, cross‑border sales, education, events, media, and creator platforms serving global audiences.

In evaluations across Hindi, Tamil, and Telugu, the model delivered 12.5% lower Word Error Rates than any other model tested, with lower fallback rates, higher task completion, and latency that sustains natural conversation.

“Building voice AI for India means handling diverse regional phonetics. In our evals across Hindi, Tamil, and Telugu, GPT-Realtime-Translate delivered 12.5% lower Word Error Rates than any other model we tested, along with lower fallback rates, higher task completion, and latency that sustained natural conversation. It sets a new standard for multilingual voice AI.” — Prateek Sachan, Co-founder & CTO at BolnaAI

GPT‑Realtime‑Whisper capabilities

GPT‑Realtime‑Whisper is a streaming speech‑to‑text model that transcribes audio as people speak, enabling low‑latency captions, live meeting notes, and continuous understanding for voice agents. It supports use cases such as meeting captions, classroom transcripts, broadcast captions, and faster follow‑up workflows in customer support, healthcare, sales, recruiting, and other high‑volume spoken interactions.

Voice interaction patterns

Developers can apply these models to three emerging voice AI patterns:

• Voice‑to‑action: users describe a need and the system reasons, uses tools, and completes the task (e.g., Zillow’s home‑search assistant).
• Systems‑to‑voice: software turns context into live spoken guidance (e.g., a travel app announcing gate changes and routes).
• Voice‑to‑voice: AI helps live conversations continue across languages, tasks, or changing context (e.g., Deutsche Telekom’s multilingual support).

These patterns can combine; Priceline is working toward a voice‑driven trip manager that searches, handles changes, provides real‑time updates, and translates conversations on the ground.

Safety and usage

The Realtime API incorporates active classifiers that can halt sessions violating harmful content guidelines. Developers may add additional guardrails via the Agents SDK. Usage policies prohibit repurposing outputs for spam, deception, or other harmful purposes, and developers must disclose AI interaction unless it is already obvious from context.

Pricing and availability

GPT‑Realtime‑2 is priced at $32 per 1M audio input tokens ($0.40 for cached input tokens) and $64 per 1M audio output tokens. GPT‑Realtime‑Translate is priced at $0.034 per minute. GPT‑Realtime‑Whisper is priced at $0.017 per minute. All three models are available now in the Realtime API.

Getting started

To begin, open the provided Codex prompt to add or extend a minimal Realtime‑2 WebRTC voice agent, or download the Codex app if not already installed. The prompt includes instructions for setting up a server‑side session endpoint, connecting microphone and audio output via RTCPeerConnection, and registering a sample tool such as check_calendar.