OpenAI Responses API WebSocket Mode Release

OpenAI has introduced WebSocket support to the Responses API to accelerate agentic workflows. By replacing synchronous HTTP calls with a persistent connection, OpenAI has reduced end-to-end latency for agent loops by up to 40%, allowing users to leverage the high-speed inference of models like GPT-5.3-Codex-Spark.

Solving the API Bottleneck in Agentic Workflows

As model inference speeds increase, the overhead of the API services—such as request validation, processing, and network hops—becomes the primary bottleneck in agentic loops. In an agentic workflow, a model may perform dozens of back-and-forth requests to determine actions, execute tools, and process results.

Previously, flagship models like GPT-5 and GPT-5.2 operated at approximately 65 tokens per second (TPS). However, with the launch of GPT-5.3-Codex-Spark, which utilizes specialized Cerebras hardware, the target speed was over 1,000 TPS. At this scale, the cumulative overhead of treating each request as independent—re-processing conversation state and history for every follow-up—created a structural delay that negated the speed gains of the GPUs.

Technical Optimizations for Latency Reduction

Before implementing WebSockets, OpenAI conducted a performance sprint in November 2025 to reduce critical-path latency for single requests. These optimizations included:

• Memory Caching: Caching rendered tokens and model configurations to avoid expensive tokenization and network calls for multi-turn responses.
• Network Hop Reduction: Eliminating intermediate service calls (e.g., image processing resolution) to call the inference service directly.
• Safety Stack Improvements: Optimizing classifiers to flag conversations more quickly.

While these changes improved time to first token (TTFT) by nearly 45%, they were insufficient for the 1,000+ TPS requirements of the newest coding models.

Implementation of WebSocket Persistent Connections

To eliminate redundant work, OpenAI transitioned from synchronous HTTP requests to a persistent WebSocket connection. This allows the API to cache reusable state in memory for the duration of the connection rather than rebuilding the conversation context from scratch for every request.

API Design and State Management

OpenAI chose WebSockets over gRPC bidirectional streaming to maintain a developer-friendly experience and keep existing input and output shapes. The final implementation allows developers to continue using response.create with the previous_response_id parameter.

When a previous_response_id is provided over a WebSocket connection, the server retrieves the following from a connection-scoped, in-memory cache:

• The previous response object.
• Prior input and output items.
• Tool definitions and namespaces.
• Reusable sampling artifacts, such as previously rendered tokens.

Resulting Performance Gains

This state-reuse architecture enables several specific technical efficiencies:

• Incremental Processing: Safety classifiers and request validators only process new input instead of the full history.
• Tokenization Efficiency: In-memory caches of rendered tokens are appended to, skipping unnecessary re-tokenization.
• Routing Optimization: Model resolution and routing logic are reused across requests.
• Asynchronous Billing: Non-blocking post-inference work, such as billing, can overlap with subsequent requests.

Production Impact and Benchmarks

Following an alpha period with coding agent startups, WebSocket mode was rolled out to production. The results demonstrate that the Responses API can now keep pace with ultra-fast inference hardware:

• Throughput: GPT-5.3-Codex-Spark hit the 1,000 TPS target, with bursts reaching up to 4,000 TPS.
• Vercel AI SDK: Reported latency decreases of up to 40%.
• Cline: Multi-file workflows became 39% faster.
• Cursor: OpenAI models became up to 30% faster.
• General Adoption: Codex has transitioned the majority of its Responses API traffic to WebSocket mode, benefiting users of GPT-5.3-Codex, GPT-5.4, and newer models.