OpenAI Embedded Assessments Phase One Overview
Overview of Embedded Assessments
OpenAI has initiated a new phase of safety evaluation by granting four independent assessors—Apollo Research, METR, Transluce, and Goodfire—deep access to its internal models, training data, and monitoring systems. This program is designed to provide independent verification of model alignment and the effectiveness of safety safeguards, supporting the company's broader safety cases and commitments under the White House Accord on Super Intelligence.
Partner Organizations and Focus Areas
Each assessor brings a distinct focus to the partnership, ranging from subversion detection to interpretability and evaluation quality.
Apollo Research
Apollo Research will investigate whether models can covertly act against developer intentions. Their assessment covers four primary claims, including whether training incentivizes subversion and whether current red teaming techniques are sufficient to detect such behaviors.
Goodfire
Goodfire will focus on training run monitoring and alignment. By embedding interpretability researchers and utilizing their proprietary platform, they aim to provide actionable insights to OpenAI’s internal research teams to improve safety during the training process.
METR
METR is conducting two specific assessments. The first focuses on model alignment within OpenAI’s largest reinforcement learning (RL) runs to date, specifically looking for failures missed by existing monitors. The second is a continuation of previous work involving monitor stress testing and control assessments.
Transluce
Transluce will evaluate the quality and coverage of OpenAI’s post-training alignment evaluations. This initial phase will be followed by a more extensive engagement in early 2027, which will include deeper access to information regarding how evaluation results influence model release decisions.
Operational Protocols and Security
To balance transparency with the protection of sensitive intellectual property, OpenAI has established specific protocols for these engagements:
- On-site Environment: Assessors requiring access to sensitive information must work on-site at OpenAI offices using company-provided hardware to ensure security standards are maintained.
- Oversight: The Safety and Security Committee (SSC) of the OpenAI Foundation’s Board has full visibility into the process, including access to unredacted reports. Disputes between OpenAI and assessors are escalated to the SSC for final resolution.
- Employee Engagement: Discussions between assessors and OpenAI staff are conducted off-the-record to encourage candor. Assessors may request "attested statements" from OpenAI to verify or clarify information learned during these discussions.
- Publication: While assessors maintain independence in their reporting, OpenAI retains the right to redact confidential information and request responsible disclosure delays to remediate identified security or safety risks.
Future Outlook
This first phase of embedded assessments is intended to establish a foundation for more consistent, long-term third-party oversight. OpenAI plans to expand these efforts to cover a broader range of safety questions, including the evaluation of organizational culture, internal processes, and compliance with established safety frameworks.