OpenAI Fine-Tuning API Improvements and Custom Models Program Expansion

OpenAI has updated its fine-tuning API with enhanced developer controls and expanded its Custom Models program to provide assisted fine-tuning and fully custom-trained models for organizations requiring deep domain expertise.

New Fine-Tuning API Features

OpenAI has introduced several features to the self-serve fine-tuning API to provide developers with more granular control over training jobs and model evaluation. These updates aim to help developers reduce latency, improve accuracy, and lower costs by optimizing how models are trained for specific tasks.

Key technical improvements include:

  • Epoch-based Checkpoint Creation: The API now automatically produces one full fine-tuned model checkpoint during each training epoch. This reduces the need for retraining, particularly when addressing overfitting.
  • Comparative Playground: A new side-by-side UI allows developers to perform human evaluation by comparing the outputs of multiple models or fine-tune snapshots against a single prompt.
  • Third-party Integration: Support for third-party platforms has been added, starting with Weights and Biases, enabling developers to integrate detailed fine-tuning data into their existing technical stacks.
  • Comprehensive Validation Metrics: Developers can now compute loss and accuracy metrics over the entire validation dataset rather than relying on a sampled batch, providing a more accurate view of model quality.
  • Hyperparameter Configuration: Hyperparameters can now be configured directly from the Dashboard, removing the requirement to use only the API or SDK.
  • Dashboard Improvements: The fine-tuning dashboard now supports rerunning jobs from previous configurations and viewing more detailed training metrics.

Case Study: Indeed

Indeed utilized the fine-tuning API for GPT-3.5 Turbo to generate personalized job recommendations. By fine-tuning the model to produce higher quality explanations, Indeed reduced prompt tokens by 80%, which allowed them to scale from under one million messages per month to approximately 20 million.

Expanded Custom Models Program

For organizations with needs beyond the self-serve API, OpenAI has expanded its Custom Models program. This program provides direct collaboration with OpenAI researchers to optimize models for specific domains using advanced techniques.

Assisted Fine-Tuning

Assisted fine-tuning is a collaborative offering that employs techniques beyond the standard API, including advanced hyperparameters and various parameter-efficient fine-tuning (PEFT) methods at scale. This service is designed for organizations that require help establishing efficient training data pipelines, evaluation systems, and bespoke parameters.

Example: SK Telecom SK Telecom worked with OpenAI to fine-tune GPT-4 for the telecommunications domain in the Korean language. This collaboration resulted in:

  • A 35% increase in conversation summarization quality.
  • A 33% increase in intent recognition accuracy.
  • An increase in customer satisfaction scores from 3.6 to 4.5 (out of 5) compared to the base GPT-4 model.

Custom-Trained Models

For organizations with massive proprietary datasets (millions of examples or billions of tokens), OpenAI offers fully custom-trained models. These models are built from scratch by modifying the model training process through novel mid-training and post-training techniques to imbue the model with deep domain knowledge.

Example: Harvey Harvey, a legal AI tool, partnered with OpenAI to create a custom-trained model for case law. After determining that prompt engineering and RAG were insufficient for the required depth of legal history, Harvey integrated the equivalent of 10 billion tokens of data. OpenAI modified the training process, including domain-specific mid-training and incorporating expert attorney feedback. This resulted in:

  • An 83% increase in factual responses.
  • A 97% preference rate for the customized model's outputs over GPT-4 among attorneys.

Platform Status Update

As of May 8, 2026, OpenAI is winding down the fine-tuning platform. The platform is no longer accessible to new users, though existing users can continue to create training jobs for several months. All fine-tuned models will remain available for inference until their respective base models are deprecated.

Sources