OpenAI New Embedding Models and API Updates January 2024

OpenAI has introduced two new embedding models, updated versions of GPT-4 Turbo and GPT-3.5 Turbo, and new API management tools to reduce costs and improve model performance for developers.

New Text Embedding Models

OpenAI has released text-embedding-3-small and text-embedding-3-large, both of which offer performance improvements over the previous text-embedding-ada-002 model.

text-embedding-3-small

text-embedding-3-small is designed for high efficiency and provides a significant upgrade in retrieval performance while reducing costs.

  • Performance Gains: On the MIRACL benchmark for multi-language retrieval, the average score increased from 31.4% (ada-002) to 44.0%. On the MTEB benchmark for English tasks, the score increased from 61.0% to 62.3%.
  • Cost Reduction: Pricing has been reduced by 5X compared to text-embedding-ada-002, dropping from $0.0001 to $0.00002 per 1k tokens.

text-embedding-3-large

text-embedding-3-large is the highest-performing embedding model available from OpenAI, supporting up to 3072 dimensions.

  • Performance Gains: On the MIRACL benchmark, the average score increased from 31.4% (ada-002) to 54.9%. On the MTEB benchmark, the score increased from 61.0% to 64.6%.

Native Embedding Shortening

Both new embedding models support a dimensions API parameter that allows developers to shorten embeddings by removing numbers from the end of the sequence without losing the core concept-representing properties. This enables a trade-off between performance and the cost of compute, memory, and storage.

For example, a text-embedding-3-large embedding shortened to 256 dimensions still outperforms an unshortened text-embedding-ada-002 embedding of 1536 dimensions on the MTEB benchmark.

GPT Model Updates and Pricing

OpenAI has updated its primary LLM offerings to improve accuracy and reduce operational costs for developers.

GPT-3.5 Turbo (gpt-3.5-turbo-0125)

Launched in early 2024, this model includes higher accuracy for requested formats and a fix for a text encoding bug affecting non-English language function calls.

  • Price Reductions: Input prices are reduced by 50% to $0.0005 /1K tokens, and output prices are reduced by 25% to $0.0015 /1K tokens.
  • Deployment: Users of the unpinned gpt-3.5-turbo alias are automatically upgraded from gpt-3.5-turbo-0613 to gpt-3.5-turbo-0125 two weeks after launch.

GPT-4 Turbo Preview (gpt-4-0125-preview)

The updated GPT-4 Turbo preview is designed to be more thorough in tasks like code generation and to reduce instances of "laziness" where the model fails to complete a task. It also includes the fix for the non-English UTF-8 generation bug.

OpenAI has introduced the gpt-4-turbo-preview alias, which always points to the latest GPT-4 Turbo preview model. General availability for GPT-4 Turbo with vision is expected in the coming months.

Safety and Platform Improvements

Updated Moderation Model

OpenAI released text-moderation-007, its most robust moderation model to date. The text-moderation-latest and text-moderation-stable aliases now point to this version.

API Key and Usage Management

Two new platform features provide developers with more granular control and visibility:

  • API Key Permissions: Developers can now assign specific permissions to API keys, such as read-only access or restrictions to specific endpoints.
  • Key-Level Usage Metrics: After enabling tracking, the usage dashboard and export functions now provide metrics at the API key level, allowing developers to track usage by feature, team, product, or project.

Sources