gpt-oss-safeguard release notes / what's new

OpenAI has released a research preview of gpt-oss-safeguard, a set of open-weight reasoning models specifically designed for safety classification. These models are available in two sizes—gpt-oss-safeguard-120b and gpt-oss-safeguard-20b—and are distributed under the permissive Apache 2.0 license via Hugging Face.

Reasoning-Based Safety Classification

gpt-oss-safeguard shifts the safety classification paradigm from training classifiers to infer a decision boundary from labeled examples to a reasoning-based approach. Instead of having policies baked into the weights during training, the model interprets a developer-provided policy directly at inference time.

Key Technical Capabilities

  • Dynamic Policy Application: Developers can provide a policy and content to classify simultaneously. The model uses this policy to determine if the content violates the specific rules provided, making it highly flexible for evolving risks or niche domains.
  • Chain-of-Thought Reasoning: The model outputs a conclusion along with the reasoning used to reach that decision. This provides transparency and allows developers to review how the model is interpreting the policy.
  • Iterative Policy Refinement: Because policies are provided at inference, developers can revise their safety guidelines without the need to retrain the model.

Ideal Use Cases

This reasoning-based approach is particularly effective in scenarios where:

  • Potential harms are emerging or evolving rapidly, requiring quick policy adaptations.
  • The domain is highly nuanced, making it difficult for smaller, traditional classifiers to handle.
  • There is insufficient training data to build a high-quality classifier for every specific risk.
  • Explainability and high-quality labels are prioritized over low latency.

Comparison with Traditional Classifiers

Traditional safety classifiers, such as those used in the OpenAI Moderation API, are typically trained on thousands of manually curated examples of safe and unsafe content. While these traditional classifiers offer low latency and lower operating costs, they require significant time and data to develop and update.

In contrast, gpt-oss-safeguard allows developers to apply any policy—including custom-written ones—and leverages reasoning to generalize across new policies, extending its utility beyond safety to general content labeling for specific product needs.

Internal Implementation: The Safety Reasoner

gpt-oss-safeguard is an open-weight implementation of the internal tool OpenAI uses called Safety Reasoner. This tool was developed using reinforcement fine-tuning on policy labeling tasks, rewarding the model for mirroring human expert judgments.

OpenAI utilizes Safety Reasoner in several ways within its own infrastructure:

  • Iterative Deployment: New models are often deployed with strict policies and high compute allocation for Safety Reasoner to carefully apply them, with policies adjusted as production risks are understood.
  • Real-time Blocking: For image generation and Sora 2, it performs step-wise evaluations to block unsafe generations in real time.
  • Multi-layered Defense: In domains like biology and self-harm, fast, high-recall classifiers are used first to identify content of interest, which is then reviewed by Safety Reasoner for a final decision.
  • System Integration: It forms part of the safeguards for GPT-5 and ChatGPT Agent, classifying outputs against a detailed taxonomy.

Performance and Benchmarks

OpenAI evaluated gpt-oss-safeguard on internal and external sets:

  • Multi-policy Accuracy: gpt-oss-safeguard and the internal Safety Reasoner outperformed both gpt-5-thinking and the gpt-oss open models on multi-policy accuracy.
  • 2022 Moderation Evaluation Set: gpt-oss-safeguard slightly outperformed all other tested models, including gpt-5-thinking and the internal Safety Reasoner.
  • ToxicChat: On this set, the internal Safety Reasoner and gpt-5-thinking marginally outperformed gpt-oss-safeguard, though OpenAI notes that the smaller size of gpt-oss-safeguard makes it a preferable choice for many tasks.

Limitations

There are two primary limitations to the reasoning-based approach:

  1. Performance Gap: Classifiers trained on tens of thousands of high-quality labeled samples can still outperform reasoning-based models on complex risks.

  2. Compute and Latency: The process is more compute-intensive and slower than traditional classifiers. OpenAI manages this internally by using smaller classifiers to filter content first or by running Safety Reasoner asynchronously to maintain a low-latency user experience.

Community Collaboration

OpenAI developed gpt-oss-safeguard in collaboration with trust and safety specialists from Discord, Tomoro, SafetyKit, and ROOST. As part of this launch, ROOST is establishing the ROOST Model Community (RMC) on GitHub to share best practices for implementing open-source AI safety models into workflows.

Sources