OpenAI Security Collaborations with US CAISI and UK AISI

OpenAI has established voluntary partnerships with the US Center for AI Standards and Innovation (CAISI) and the UK AI Security Institute (UK AISI) to improve the security of frontier AI systems. These collaborations focus on joint red-teaming, end-to-end product testing, and the creation of rapid feedback loops to resolve vulnerabilities in agentic AI and biosecurity safeguards.

Securing Agentic AI Systems with US CAISI

OpenAI partnered with CAISI to evaluate the security of agentic AI systems, specifically focusing on the ChatGPT Agent product. This collaboration involved a multidisciplinary team of cybersecurity and AI agent security experts who were granted early access to the system architecture to identify vulnerabilities.

Discovery of Complex Exploit Chains

CAISI identified two novel security vulnerabilities in ChatGPT Agent. While initially appearing unexploitable, CAISI discovered that combining these traditional software vulnerabilities with an AI agent hijacking attack could bypass AI-based security protections. This resulted in a full exploit chain with a success rate of approximately 50%.

If successful, these attacks could have allowed a sophisticated attacker to:

  • Remotely control computer systems accessed by the agent during a session.
  • Impersonate the user on other websites where the user was logged in.

OpenAI reported that these vulnerabilities were fixed within one business day of being reported.

Strengthening Biosecurity Safeguards with UK AISI

In May, the UK AI Security Institute (UK AISI) began red-teaming OpenAI's safeguards against biological misuse, covering both GPT-5 and ChatGPT Agent. This is an ongoing collaboration designed to continuously improve the safeguards stack rather than being tied to a single launch.

Deep System Access for Rigorous Testing

To facilitate thorough evaluation, UK AISI was granted high-level access to non-public resources, including:

  • Non-public prototypes of safeguard systems.
  • "Helpful-only" model variants with guardrails removed.
  • Internal policy guidance on biological misuse.
  • Access to the chain of thought (CoT) of internal safety monitor models.
  • The ability to selectively disable mitigations and enforcement during testing.

Results and Iterative Improvements

UK AISI identified more than a dozen detailed vulnerability reports. These findings led to several types of improvements:

  • Engineering fixes: Direct product fixes based on vulnerability reports.
  • Policy enforcement: Implementation fixes for policy enforcement.
  • Policy training: Focused training to improve classifiers.
  • Monitoring robustness: Improvements to the monitoring stack to defend against universal jailbreaks identified by UK AISI.
  • Configuration fixes: Resolution of configuration vulnerabilities where malicious content could be exfiltrated or inputted without triggering moderation.

UK AISI concluded that the full moderation system's safeguards were substantially strengthened. By the end of the iteration period, creating universal attacks required sophisticated techniques and a large number of moderation flags, meaning malicious actors would likely be flagged and banned.

Strategic Implications of Public-Private Security Partnerships

These collaborations demonstrate a model for the deep integration of government national security expertise with private AI development. OpenAI notes that these partnerships provide several key benefits:

  • National Security Expertise: Access to government expertise in machine learning and metrology allows for the identification of issues that the others might miss.
  • Technical Grounding: Hands-on experience allows governments to develop technically-grounded best practices that can accelerate industry-wide security.
  • Accountability and Trust: External expert analysis builds trust and accountability in theployed systems.
  • Continuous Evaluation: Ongoing collaborations provide more value than one-off pre-deployment evaluations.

Sources