Advancing red teaming with people and AI – OpenAI's approach and application to the o1 family
TL;DR
OpenAI released a white paper that outlines its four‑step method for external red teaming of AI models and described how this method was used to test the OpenAI o1 family before public release.
Approach to External Red Teaming
The white paper describes a repeatable process for designing effective red teaming campaigns.
1. Choosing the composition of the red teaming group
Red team composition is based on the goals and key testing areas of the model. Teams include people with diverse perspectives such as expertise in natural sciences, cybersecurity, regional political knowledge, or languages spoken. Threat modeling precedes the exercise to prioritize testing areas, considering expected model capabilities, previously observed issues, and potential applications. Internal teams set initial priorities, which external red teamers later refine or expand.
2. Deciding which model or system versions red teamers access
The version of the model made available to red teamers can influence outcomes and should align with campaign goals. Early‑access testing without safety mitigations can reveal new risks from increased capabilities, while later versions can test planned mitigations. Red teamers may test multiple versions throughout the campaign.
3. Creating interfaces, instructions, and documentation guidance
Effective interaction relies on clear instructions, suitable testing interfaces, and actionable documentation. Instructions cover model descriptions, existing or planned safeguards, how to use the interface, prioritized testing areas, and guidelines for documenting results. Interfaces may be APIs or consumer product interfaces such as ChatGPT, enabling rapid programmatic testing, feedback collection on specific prompts, or simulation of user interactions. Structured feedback from these interfaces can later inform risk assessments and automated evaluations.
4. Synthesizing data and creating evaluations
After the campaign, red team outputs are assessed for quality and mapped to existing policies to determine whether they violate policy, require new policy, or need behavior changes. Data that passes quality checks can be turned into repeatable, automated evaluations for future model updates.
Application to the OpenAI o1 family
OpenAI applied this approach to prepare the OpenAI o1 family of models for public use. The external red teaming campaign tested the models for:
- Resistance to jailbreaks
- Safe handling of real‑world attack planning prompts
- Safe application in natural sciences
- Broader topics such as AI research and development capabilities
The campaign followed the four steps outlined above, using appropriate team composition, model versions, interfaces, and documentation to generate data that fed into policy review and automated evaluation pipelines.
Note: All details are drawn directly from the white paper and the accompanying announcement; no external claims, numbers, or quotes have been added.