OpenAI gpt-oss Release Notes

OpenAI has released gpt-oss-120b and gpt-oss-20b, two state-of-the-art open-weight language models designed for high-performance reasoning and efficient deployment. Licensed under Apache 2.0, these models are optimized for consumer hardware and were trained using reinforcement learning and techniques derived from OpenAI's frontier systems, including o3.

Model Capabilities and Benchmarks

gpt-oss models provide strong real-world performance in reasoning, tool use, and instruction following, with specific strengths in STEM, coding, and general knowledge.

Performance Comparison

  • gpt-oss-120b: Achieves near-parity with OpenAI o4-mini on core reasoning benchmarks. It matches or exceeds o4-mini on competition coding (Codeforces), general problem solving (MMLU and HLE), and tool calling (TauBench). It outperforms o4-mini on competition mathematics (AIME 2024 & 2025) and health-related queries via HealthBench.
  • gpt-oss-20b: Matches or exceeds OpenAI o3-mini on the same evaluations, including outperforming o3-mini on health and competition mathematics.

Key Features

  • Reasoning Effort: Like the o-series models, gpt-oss supports three reasoning effort levels (low, medium, and high) to allow developers to trade off latency for performance via the system message.
  • Tool Use: The models demonstrate strong capabilities in few-shot function calling and agentic workflows, including Python code execution and web search.
  • Structured Outputs: Both models support Structured Outputs and are compatible with the Responses API.

Technical Architecture

Both gpt-oss models utilize a Transformer architecture with Mixture-of-Experts (MoE) to optimize the number of active parameters during processing.

Model Layers Total Params Active Params/Token Total Experts Active Experts/Token Context Length
gpt-oss-120b 36 117B 5.1B 128 4 128k
gpt-oss-20b 24 21B 3.6B 32 4 128k

Architectural Details

  • Attention Mechanism: The models use alternating dense and locally banded sparse attention patterns, combined with grouped multi-query attention (group size of 8) for memory efficiency.
  • Positional Encoding: Rotary Positional Embedding (RoPE) is used to support native context lengths of up to 128k tokens.
  • Tokenizer: The models use o200k_harmony, a superset of the tokenizer used for GPT-4o and o4-mini, which OpenAI has also open-sourced.

Post-Training and Chain-of-Thought

The models underwent a supervised fine-tuning stage followed by a high-compute reinforcement learning (RL) stage to align with the OpenAI Model Spec.

Non-Supervised Chain-of-Thought (CoT)

OpenAI did not apply direct supervision to the Chain-of-Thought (CoT) for either model. This design choice allows researchers to monitor for model misbehavior, deception, and misuse. OpenAI advises developers not to show CoTs directly to users, as they may contain hallucinations or content that violates safety policies.

Safety and Risk Mitigation

Safety was integrated into the gpt-oss release through multi-stage training and adversarial testing.

Training and Alignment

  • Pre-training: Harmful data related to Chemical, Biological, Radiological, and Nuclear (CBRN) risks were filtered out.
  • Post-training: Deliberative alignment and the instruction hierarchy were used to prevent prompt injections and ensure the refusal of unsafe prompts.

Adversarial Fine-Tuning

To assess the risk of malicious fine-tuning, OpenAI created non-refusing versions of the models using specialized biology and cybersecurity data. Testing under the Preparedness Framework indicated that these maliciously fine-tuned models were unable to reach high capability levels, even when using OpenAI's internal training stack.

Red Teaming

OpenAI is hosting a Red Teaming Challenge with a $500,000 prize fund to identify novel safety issues, with the intent to publish a report and an open-source evaluation dataset based on the findings.

Availability and Deployment

Weights are available on Hugging Face, natively quantized in MXFP4 to minimize memory requirements.

  • gpt-oss-120b: Runs within 80GB of memory (e.g., a single 80GB GPU).
  • gpt-oss-20b: Requires only 16GB of memory, making it suitable for edge devices and local inference.

Ecosystem Support

  • Software: OpenAI has open-sourced a harmony renderer in Python and Rust, and provided reference implementations for PyTorch and Apple's Metal platform.
  • Partners: Deployment is supported across major platforms including Azure, Hugging Face, vLLM, Ollama, llama.cpp, LM Studio, AWS, Fireworks, Together AI, Baseten, Databricks, Vercel, Cloudflare, and OpenRouter.
  • Hardware: Performance is optimized for NVIDIA, AMD, Cerebras, and Groq hardware.
  • Windows Integration: Microsoft is providing GPU-optimized versions of gpt-oss-20b for Windows devices via ONNX Runtime, available through Foundry Local and the AI Toolkit for VS Code.

Sources