OpenAI DevDay 2023: GPT-4 Turbo, Assistants API, and Multimodal Updates

OpenAI has introduced GPT-4 Turbo, a more capable and cost-effective model with a 128K context window, alongside a new Assistants API designed to simplify the creation of AI agents. These updates, announced at DevDay, significantly expand the platform's multimodal capabilities and reduce pricing for developers.

GPT-4 Turbo: Expanded Context and Lower Costs

GPT-4 Turbo is the next generation of the GPT-4 model, featuring a 128K context window capable of processing over 300 pages of text in a single prompt. The model has updated knowledge of world events up to April 2023 and is available via the gpt-4-1106-preview API endpoint.

Pricing and Performance

GPT-4 Turbo is significantly cheaper than its predecessor. Input tokens are 3x cheaper ($0.01 per 1k tokens) and output tokens are 2x cheaper ($0.03 per 1k tokens) compared to GPT-4.

Technical Enhancements

  • Parallel Function Calling: Developers can now call multiple functions in a single message, reducing the number of roundtrips required for complex requests.
  • JSON Mode: A new response_format parameter ensures the model generates syntactically correct JSON, improving reliability for developers using the Chat Completions API.
  • Reproducible Outputs: The introduction of a seed parameter allows for more consistent completions, which is critical for debugging and unit testing.
  • Log Probabilities: OpenAI will soon support returning log probabilities for the most likely output tokens for both GPT-4 Turbo and GPT-3.5 Turbo.

Updated GPT-3.5 Turbo

OpenAI released a new version of GPT-3.5 Turbo (gpt-3.5-turbo-1106) that supports a 16K context window by default. Internal evaluations show a 38% improvement in format-following tasks, such as generating JSON, XML, and YAML.

Assistants API: Building AI Agents

The Assistants API allows developers to build "agent-like" experiences by creating purpose-built AI with specific instructions and access to tools. A primary innovation is the use of persistent and infinitely long threads, which offloads state management to OpenAI and bypasses traditional context window constraints.

Integrated Tools

  • Code Interpreter: Enables the assistant to write and execute Python code in a sandboxed environment to solve math problems or generate charts.
  • Retrieval: Allows assistants to access proprietary domain data or user-provided documents without requiring the developer to implement their own embedding or chunking algorithms.
  • Function Calling: Allows assistants to invoke custom-defined functions and incorporate the responses into their messages.

New Multimodal Capabilities

OpenAI has expanded the API to include vision, image generation, and speech synthesis:

  • GPT-4 Turbo with Vision: Accessible via gpt-4-vision-preview, this allows the model to analyze images, generate captions, and read documents containing figures.
  • DALL·E 3: Now available via the Images API (dall-e-3), allowing developers to programmatically generate images with built-in moderation.
  • Text-to-Speech (TTS): A new API provides human-quality speech from text with six preset voices and two variants: tts-1 (optimized for real-time) and tts-1-hd (optimized for quality).

Model Customization and Fine-Tuning

OpenAI is introducing two paths for model customization:

  1. GPT-4 Fine-Tuning: An experimental access program is being established for GPT-4 fine-tuning, though preliminary results suggest it requires more effort for meaningful gains compared to GPT-3.5.
  2. Custom Models Program: A limited, high-cost program for organizations with massive proprietary datasets (billions of tokens) to work with OpenAI researchers to train a custom GPT-4 model from the pre-training stage.

Platform Infrastructure and Legal Protections

Pricing and Rate Limits

OpenAI has reduced prices across the platform. Notably, GPT-3.5 Turbo 16K input tokens are now 3x cheaper ($0.001 per 1k tokens) and output tokens are 2x cheaper ($0.002 per 1k tokens). Additionally, token-per-minute limits for all paying GPT-4 customers have been doubled.

Copyright Shield

OpenAI introduced "Copyright Shield," a commitment to defend customers and cover legal costs if they face copyright infringement claims related to the use of generally available features of ChatGPT Enterprise and the developer platform.

Open Source Releases

  • Whisper large-v3: The next version of the open-source automatic speech recognition (ASR) model with improved multilingual performance.
  • Consistency Decoder: An open-source drop-in replacement for the Stable Diffusion VAE decoder that improves the rendering of text, faces, and straight lines in images compatible with Stable Diffusion 1.0+ VAE.

Sources