OpenAI Scaling Access to Codex and Sora via Hybrid Credit System

OpenAI has introduced a real-time access engine for Codex and Sora that allows users to transition seamlessly from rate limits to purchasable credits. This hybrid system ensures that users who reach their usage caps can continue working without interruption while OpenAI maintains system performance and fairness controls.

Hybrid Access Model: Combining Rate Limits and Credits

OpenAI transitioned to a hybrid access model to resolve the limitations of traditional rate-limiting and usage-based billing. Traditional rate limits often result in a poor user experience by forcing users to stop work when limits are reached, while pure usage-based billing can discourage early exploration by charging from the first token.

The new system functions as a "decision waterfall," where the system evaluates access in a specific sequence for every request:

Rate Limits: The system first checks if the user is within their allocated rate limit.
Free Tiers: If rate limits are exceeded, the system checks for available free tier allocations.
Credits/Promotions: If previous layers are exhausted, the system checks for purchased credits or promotional balances.
Enterprise Entitlements: Finally, the system checks for specific enterprise-level permissions.

This architecture allows users to remain in a single flow without manually switching systems, making the transition to credit consumption invisible to the end user.

In-House Infrastructure for Real-Time Correctness

OpenAI developed this system in-house rather than using third-party metering platforms to meet two primary technical requirements: real-time correctness and full reconcilability.

Real-Time Decisioning

For interactive products like Codex and Sora, any delay in counting usage can lead to inconsistent balances or surprise blocks. An in-house solution allows the system to know immediately when a user has hit a limit and has credits available, ensuring a seamless transition within the same request.

Observability and Trust

To maintain user trust, OpenAI required the ability to audit exactly why a request was allowed or blocked, how much usage was consumed, and which specific balances were applied. Integrating this directly into the decision waterfall provides a level of transparency that separate billing platforms cannot offer.

Technical Architecture of the Usage and Balance System

The distributed usage and balance system is designed for synchronous access decisions through the following mechanisms:

Usage Tracking: The system tracks per-user and per-feature usage in real-time.
Window Management: It maintains rate-limit windows to smooth demand.
Balance Maintenance: It tracks real-time credit balances and debits them idempotently via a streaming asynchronous processor.

Every request follows a single evaluation path that synchronously consumes rate limits and verifies credits, returning a definitive outcome while settling credit debits asynchronously.

Ensuring Provable Billing Correctness

OpenAI prioritizes provable correctness over strict enforcement to prevent misbilling. The system utilizes three interconnected datasets to create a complete audit trail:

Product Usage Events: A record of all user activity, regardless of whether it triggered a charge.
Monetization Events: A record of what the user is charged for.
Balance Updates: A record of how the credit balance was adjusted and why.

Implementation Details for Accuracy

Idempotency Keys: Every event uses a stable idempotency key to prevent double-charging during retries, replays, or worker restarts.
Asynchronous Balance Updates: Balance updates are near-real-time but asynchronous. This creates an audit trail and allows for batch reconciliation. If this delay causes a user to overshoot their credit balance, OpenAI automatically refunds the difference.
Atomic Transactions: Credit balance decreases and balance update records are inserted in a single atomic database transaction. Because updates are serialized per account, concurrent requests cannot race to spend the same credits.
Attribution: Every balance update record contains the debit amount and a direct link back to the monetization event that triggered it.