OpenAI Introducing ChatGPT and Whisper APIs

OpenAI has launched the GPT-3.5 Turbo and Whisper APIs, providing developers with access to advanced language and speech-to-text models. This release is characterized by a 90% cost reduction for the ChatGPT model family since December, which is now passed through to API users.

GPT-3.5 Turbo API Capabilities and Pricing

The gpt-3.5-turbo model is the same model used in the ChatGPT product and is designed for both chat and non-chat use cases. Early testers have successfully migrated from text-davinci-003 to gpt-3.5-turbo with minimal prompt adjustments.

  • Pricing: The model is priced at $0.002 per 1k tokens, making it 10x cheaper than previous GPT-3.5 models.
  • Input Format: Unlike traditional GPT models that consume unstructured text, ChatGPT models use a sequence of messages with metadata, rendered internally via Chat Markup Language (ChatML).
  • Model Versioning: Developers can use the gpt-3.5-turbo endpoint for the recommended stable model or opt for specific versions, such as gpt-3.5-turbo-0301.

Whisper API for Speech-to-Text

OpenAI has made the large-v2 version of its open-source Whisper model available via API. This provides on-demand access to human-level accuracy for speech-to-text tasks with a highly-optimized serving stack for faster performance.

  • Pricing: The service is priced at $0.006 per minute.
  • Endpoints: The API provides transcriptions (for source language transcription) and translations (for transcription into English).
  • Supported Formats: The API accepts m4a, mp3, mp4, mpeg, mpga, wav, and webm formats.

Dedicated Instances for High-Volume Users

For developers requiring deeper control over system performance and model versions, OpenAI now offers dedicated instances via Azure. This allows users to pay by time period for reserved compute infrastructure rather than paying per request.

Dedicated instances are economically viable for developers processing more than 450 million tokens per day. Benefits include:

  • Full control over the instance load to balance throughput and request speed.
  • The ability to pin specific model snapshots.
  • Options to enable features such as longer context limits.

Developer-Centric Policy Updates

OpenAI has updated its API terms of service and data handling policies based on developer feedback:

  • Data Privacy: Data submitted through the API is no longer used for model training or service improvements unless an organization explicitly opts in.
  • Data Retention: A default 30-day data retention policy has been implemented, with options for stricter retention.
  • Ownership: Users now own the input and output of the models.
  • Access: The pre-launch review process has been removed to streamline developer onboarding.

Industry Implementations

Several companies have integrated these APIs into their products to enhance user experience:

  • Snap Inc: Integrated gpt-3.5-turbo to power "My AI" for Snapchat+.
  • Quizlet: Developed "Q-Chat," an adaptive AI tutor using the gpt-3.5-turbo API.
  • Instacart: Using the API to enable "Ask Instacart," allowing customers to search for food and shopping ideas.
  • Shop (Shopify): Implementing a shopping assistant that provides personalized product recommendations.
  • Speak: Utilizing the Whisper API to power an AI speaking companion for language learners.

Sources