OpenAI Instruction Hierarchy Improvements and GPT-5 Mini-R
OpenAI has developed a method to improve the instruction hierarchy of frontier LLMs, enabling models to reliably prioritize high-trust instructions over lower-trust ones. This advancement improves safety steerability and robustness against prompt-injection attacks, particularly those embedded in tool outputs.
The Instruction Hierarchy Framework
To resolve conflicts between multiple instruction sources, OpenAI models follow a defined trust hierarchy: System > Developer > User > Tool.
Under this framework, higher-priority instructions are treated as more authoritative. A model should only follow lower-priority instructions if they do not conflict with the constraints set by higher-priority roles. For example, if a system message contains a safety policy, the model must refuse a user request that violates that policy. Similarly, if a tool output contains malicious instructions, the model should ignore them rather than executing them as commands.
Challenges in Instruction Hierarchy Training
While reinforcement learning (RL) is a suitable method for teaching this hierarchy, OpenAI identified three primary pitfalls in naive RL applications:
- Instruction-Following Failures: Models may fail to resolve a conflict not because they misunderstand the hierarchy, but because the instructions themselves are too complex.
- Subjective Conflicts: Instruction conflicts can be nuanced, and using another LLM as a judge to assign rewards can introduce fallibility.
- Reward Shortcuts: Models may learn shortcuts to maximize reward, such as over-refusal, where the model refuses even benign requests to ensure safety.
The IH-Challenge Approach
OpenAI addressed these pitfalls by designing IH-Challenge, a reinforcement learning training dataset built on three core principles:
- Simplicity: Tasks are designed to be simple in terms of instruction-following to isolate the hierarchy challenge.
- Objective Grading: Tasks are objectively gradable using simple Python scripts, removing the need for fallible LLM judges.
- No Trivial Shortcuts: The environment is structured to prevent models from achieving high rewards through simple shortcuts.
Each task consists of a conversation where a high-privilege role provides an instruction (e.g., "Only answer 'Yes' or 'No'") and a lower-privilege role attempts to induce the model to violate that instruction. The model's response is then programmatically checked for compliance with the higher-level constraint.
GPT-5 Mini-R: Performance and Robustness
OpenAI used IH-Challenge to train an internal model called GPT-5 Mini-R. This model demonstrates improved performance across several benchmarks without collapsing into over-refusal.
Academic Benchmark Results
| Eval | GPT-5-Mini | GPT-5 Mini-R | Improvement |
|---|---|---|---|
| Gandalf Password (sys-user) | 0.99 | 0.99 | +0 |
| Gandalf Password (dev-user) | 0.98 | 1.00 | +0.02 |
| TensorTrust (sys-user) | 0.86 | 0.94 | +0.08 |
| TensorTrust (dev-user) | 0.76 | 0.91 | +0.15 |
| RealGuardrails (Distractors) | 0.88 | 0.95 | +0.07 |
| RealGuardrails (Handwritten) | 0.82 | 0.89 | +0.07 |
| System IFEval | 0.92 | 0.96 | +0.04 |
Internal Benchmark Results
| Eval | GPT-5-Mini | GPT-5 Mini-R | Improvement |
|---|---|---|---|
| TutorJailbreak (sys-user) | 0.96 | 0.99 | +0.03 |
| Tutor Jailbreak (dev-user) | 0.97 | 0.99 | +0.02 |
| System <> User Conflict | 0.84 | 0.95 | +0.11 |
| System <> Developer Conflict | 0.86 | 0.86 | +0 |
| Developer <> User Conflict | 0.83 | 0.95 | +0.12 |
Capability Maintenance
GPT-5 Mini-R maintains general capabilities with minimal regressions. While there was a slight decrease in Chat WinRate vs. o1 (-0.05) and Preference Score (-0.06), it showed a significant improvement in IH-Challenge over-refusal (from 0.79 to 1.00).
Real-World Safety and Security Implications
Improving the instruction hierarchy provides two primary safety benefits:
Safety Steerability
By adding category-specific safety specifications to the system prompt, GPT-5 Mini-R achieves higher refusal and safe completion rates across disallowed categories on OpenAI's safety Production Benchmarks. This improvement occurs without a decrease in the overall helpfulness rate, meaning the model is not simply refusing more requests overall, but is better at resolving conflicts when unsafe requests come from lower-priority instructions.
Prompt Injection Robustness
GPT-5 Mini-R shows increased resistance to malicious instructions embedded in tool outputs. Evaluations on the academic benchmark CyberSecEval 2 and OpenAI's internal prompt injection benchmarks show that the IH-trained model substantially improves performance over the baseline GPT-5 Mini.
Conclusion and Dataset Release
As AI systems become more autonomous and agentic, the ability to prioritize trusted instructions is a critical safety property. OpenAI has released the IH-Challenge dataset to support further research in this area.