openai/gpt-oss

gpt‑oss – Open‑weight LLMs from OpenAI

What it is – A repository that ships reference code for running OpenAI’s newly released open‑weight language models, gpt‑oss‑120b (≈117 B parameters, 5.1 B active) and gpt‑oss‑20b (≈21 B parameters, 3.6 B active). The models are meant for high‑level reasoning, tool‑use (browser, Python), and agentic tasks. The repo does not contain the model weights themselves; those are hosted on Hugging Face and can be downloaded with the HF CLI.

Key ideas

Harmony response format – a structured chat schema that the models were trained on. All inference code expects prompts and outputs to follow this format; otherwise the model will not behave correctly.
MXFP4 quantization – post‑training quantization that shrinks the MoE weights so the 120 B model fits on a single 80 GB GPU (e.g., H100, MI300X) and the 20 B model fits in ~16 GB.
Apache‑2.0 license – permissive, allowing commercial use and modification without copyleft.

Highlights (from the README)

Feature	What it means
Configurable reasoning effort	Choose low/medium/high effort at inference time to trade latency for depth of reasoning.
Full chain‑of‑thought	The model returns its internal reasoning steps (not intended for end‑users) which helps debugging and trust.
Fine‑tunable	The weights can be further fine‑tuned on custom data via standard PyTorch pipelines.
Agentic capabilities	Native support for function calling, web‑browsing, Python execution, and structured outputs, all expressed through the Harmony format.
Quantization (MXFP4)	Enables the 120 B model on a single 80 GB GPU and the 20 B model on 16 GB, with the same evaluation quality as the full‑precision checkpoint.
Multiple back‑ends	Reference implementations for PyTorch (educational), Triton (single‑GPU optimized), Metal (Apple Silicon), plus ready‑to‑use wrappers for vLLM, Transformers, Ollama, and LM Studio.

How to get the model

# 120 B
hf download openai/gpt-oss-120b --include "original/*" --local-dir gpt-oss-120b/
# 20 B
hf download openai/gpt-oss-20b --include "original/*" --local-dir gpt-oss-20b/

The weights are stored as SafeTensors on the Hugging Face Hub. For Apple Silicon you can also download pre‑converted Metal binaries.

Quick inference with 🤗 Transformers

from transformers import pipeline
model_id = "openai/gpt-oss-120b"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype="auto",
    device_map="auto",
)
messages = [{"role": "user", "content": "Explain quantum mechanics clearly and concisely."}]
out = pipe(messages, max_new_tokens=256)
print(out[0]["generated_text"][-1])

The pipeline automatically applies the Harmony chat template; if you call model.generate directly you must format the prompt yourself or use the openai‑harmony package.

Running the reference back‑ends

Backend	How to install	Typical hardware
PyTorch (reference)	`pip install -e "[torch]"`	4× H100 (inefficient, educational)
Triton (single‑GPU)	Build Triton from source, then `pip install -e "[triton]"`	1× 80 GB GPU (H100/MI300X)
Metal (Apple Silicon)	`GPTOSS_BUILD_METAL=1 pip install -e "[metal]"`	Apple M‑series chips
vLLM	`uv pip install --pre vllm==0.10.1+gptoss …`	Any GPU supported by vLLM
Ollama	`ollama pull gpt-oss:20b` (or `:120b`)	Consumer‑grade CPUs/GPUs via Ollama runtime
LM Studio	`lms get openai/gpt-oss-20b`	Same as Ollama

Each implementation comes with a small CLI (python -m gpt_oss.generate …) and a terminal chat example that demonstrates tool use (browser, Python) and the Harmony format.

Tools shipped with the repo

Browser tool – a minimal web‑search / page‑fetch interface used during model training. Two back‑ends are provided (YouComBackend and ExaBackend). The code is deliberately simple and marked educational only; production systems should replace it with a secure, sandboxed browser service.
Python tool – a stateless executor that runs arbitrary Python snippets. Again, intended for research demos; sandboxing is the user’s responsibility.

Both tools expose a small JSON‑based protocol that the model can call via the Harmony messages.

Client examples

Terminal chat (gpt_oss.chat) – interactive REPL that lets you toggle reasoning effort, enable the browser or Python tool, and pick the inference backend.
Responses API server (gpt_oss.responses_api.serve) – a lightweight server mimicking OpenAI’s Responses API, useful for integrating the model with existing chat‑completion front‑ends.
Codex integration – a tiny config snippet shows how to point the open‑source Codex client at a locally‑served gpt‑oss endpoint (e.g., via Ollama).

Licensing & contribution

License: Apache 2.0 – free for commercial and research use, no viral copyleft.
Contributing: The repo follows the usual OpenAI open‑source contribution model (pull requests, code‑style checks). The reference implementations are deliberately un‑optimised; contributions that improve performance or add production‑grade tooling are especially welcome.

When to use this repo

Research & prototyping – you want a state‑of‑the‑art, open‑weight LLM that you can inspect, fine‑tune, or embed in custom agents.
Tool‑augmented agents – the built‑in browser and Python tools let you experiment with retrieval‑augmented generation or code‑execution loops.
Performance exploration – the Triton and Metal back‑ends showcase how MXFP4 quantization can shrink a 120 B MoE model to fit a single GPU.
Learning – the PyTorch reference code is intentionally simple, making it a good teaching resource for MoE architectures and token‑level inference pipelines.

Bottom line: openai/gpt-oss is a genuine, production‑oriented open‑weight LLM project. It provides the models, a structured chat format (Harmony), reference inference code for several hardware stacks, and example clients that together make it possible to run, fine‑tune, and build agentic applications on the 120 B and 20 B models without any proprietary OpenAI API.