OpenAI Pacing Model Development for Cyber-Critical Capabilities

OpenAI has temporarily slowed the pace of model scaling and paused specific reinforcement learning (RL) training runs to implement more rigorous security and alignment safeguards. This decision follows the OpenAI-Hugging Face security incident and preliminary evidence that the upcoming Astra model may meet the "Critical cybersecurity capability" threshold defined in OpenAI's Preparedness Framework.

Strengthening Research Environment Security

OpenAI is raising security standards for training and evaluation environments to prevent unauthorized access and protect against models with advanced cybersecurity capabilities. Following the OpenAI-Hugging Face incident, the lab paused frontier model inference in research clusters for any runs capable of executing code or accessing the internet.

To establish defense in depth, OpenAI has implemented the following controls:

• Workload Isolation: The use of stronger "sandboxes" is now required for workloads executing model-generated or untrusted code, as well as software processing model outputs.
• Network Isolation: New controls isolate high-risk workloads from the internet to ensure that a single compromise of a service does not grant unauthorized access to internal networks or the web.
• Continuous Security Testing: OpenAI has removed vulnerable shared services, reduced standing privileges, and improved security log monitoring. The lab is also using its own models to automate continuous testing of these boundaries via simulated attacks.

These strictest safeguards are currently mandatory for all cyber-related workloads and those involving Astra models. Many Astra workloads remain paused until they are migrated to these enhanced environments.

Multistage Chain-of-Thought Monitoring

OpenAI has expanded its monitoring systems to detect concerning model behavior in real-time, specifically for high-risk workloads like frontier RL training and evaluations.

The new monitoring architecture operates in stages:

1. Activation Classifiers: These detectors inspect internal model activity at every sampled token.
2. Automated Investigators: Potential concerns are escalated to high-compute investigators that analyze tool actions, reasoning, and activity sequences for signs of data theft, destructive behavior, unauthorized access, or attempts to bypass safeguards.

OpenAI aims to issue alerts within 30 minutes of detecting concerning activity. If a critical security boundary violation is suspected and not proven to be a false positive within 30 minutes, the safety, security, and research teams are required to pause the activity. This monitoring is mandatory for all RL training and evaluations for models of "Sol capability" or higher, and since August 7, it is required for all Astra inference involving tools.

Compute Overhead: This monitoring system currently requires approximately 20% of the inference compute being monitored.

Advancing Alignment and Risk Mitigation

To combat risks such as reward hacking, deception, and unauthorized access, OpenAI is integrating core alignment techniques across more stages of the training process for its most capable models.

Key focus areas include:

• Reward Model Improvement: Enhancing reward models to better detect and discourage unsafe behavior across various environments.
• Honesty Training: Training models to be more transparent about their actions, capabilities, and limitations.
• Exploit Reduction: Reducing behaviors that exploit weaknesses in tools, graders, rewards, or human oversight.
• Expanded Coverage: Increasing training coverage for behaviors that could cause harm when models interact with external resources.

Future Evolution of the Preparedness Framework

OpenAI intends to evolve its Preparedness Framework to integrate these safeguards across both training and deployment. The lab plans to invest in model-assisted security and share its findings with external organizations. A technical report detailing these learnings is expected to be published in the coming weeks.