OpenAI Enterprise API Feature Updates April 2024

OpenAI has released a suite of enterprise-grade features for its API customers designed to improve security, administrative oversight, and cost efficiency for organizations scaling AI deployments. These updates introduce Private Link for secure Azure communication, a new Projects management system, significant scaling for the Assistants API, and discounted pricing for committed and asynchronous workloads.

Enhanced Enterprise Security and Compliance

OpenAI has expanded its security stack to minimize internet exposure and improve access control for enterprise users.

  • Private Link: A new connectivity option that enables direct communication between Azure and OpenAI, reducing exposure to the open internet.
  • Multi-Factor Authentication (MFA): Native MFA is now available to meet stricter organizational access control requirements.
  • Existing Security Framework: These additions complement existing certifications and protocols, including SOC 2 Type II certification, single sign-on (SSO), AES-256 encryption at rest, and TLS 1.2 encryption in transit.
  • Healthcare Compliance: OpenAI continues to offer Business Associate Agreements (BAAs) for HIPAA compliance and zero data retention policies for qualifying API use cases.

Granular Administrative Control via Projects

The new "Projects" feature allows organizations to implement more precise oversight and resource allocation across different internal initiatives.

  • Scoped Access: Administrators can now scope roles and API keys to specific projects and restrict or allow specific models for those projects.
  • Resource Management: Project-level usage and rate-based limits can be set to prevent unexpected budget overages.
  • Service Accounts: Project owners can create service account API keys, allowing project access that is not tied to a specific individual user account.

Assistants API Technical Improvements

Updates to the Assistants API focus on increasing the scale of data retrieval, improving response latency, and providing finer control over model behavior.

Retrieval and Data Management

  • file_search Scaling: The retrieval tool now supports up to 10,000 files per assistant, representing a 500x increase from the previous limit of 20. This tool features multi-threaded searches for parallel queries, as well as improved reranking and query rewriting.
  • vector_store Objects: New vector store objects automate the parsing, chunking, and embedding of files. These stores can be shared across multiple assistants and threads to simplify billing and file management.

Execution and Control

  • Streaming Support: The API now supports streaming for real-time conversational responses.
  • tool_choice Parameter: Developers can now explicitly select a specific tool (such as code_interpreter, file_search, or a specific function) for a particular run.
  • Fine-tuned Model Support: The API now supports fine-tuned GPT-3.5 Turbo models, specifically starting with gpt-3.5-turbo-0125.

Cost Control

  • Token Limits: Users can now set a maximum number of tokens per run and limit the number of previous and recent messages used in each run to manage costs.

Enterprise Cost Management Options

OpenAI has introduced two new pricing models to reduce costs for high-volume and non-urgent workloads.

  • Provisioned Throughput: Customers with sustained tokens per minute (TPM) usage on GPT-4 or GPT-4 Turbo can request provisioned throughput, which offers discounts between 10% and 50% depending on the commitment size.
  • Batch API: For non-urgent tasks—such as synthetic data generation, offline classification, summarization, and model evaluation—the Batch API allows asynchronous processing. Batch API requests are priced at 50% off shared prices, provide higher rate limits, and return results within 24 hours.

Sources