OpenAI Preparing for Malicious Uses of AI

OpenAI, in collaboration with the Future of Humanity Institute, the Centre for the Study of Existential Risk, the Center for a New American Security, the Electronic Frontier Foundation, and other partners, has published a research paper forecasting the potential misuse of AI technology by malicious actors. This work aims to provide a framework for preventing and mitigating threats to global security caused by the lowering of attack costs, the creation of new vulnerabilities, and the increased difficulty of attack attribution.

AI as a Global Security Challenge

AI technology presents a security risk because it reduces the cost of executing existing attacks and introduces new vulnerabilities while complicating the attribution of specific attacks. The report identifies AI as a dual-use technology, meaning it can be used for both immensely positive and immensely negative applications.

Dual-Use Nature and Risk Assessment

Because AI tools can be used for both public good and harm, the community must evaluate research projects for potential perversion by malicious actors. Examples of dual-use risks include:

  • Surveillance tools: These can be used to catch terrorists or to oppress ordinary citizens.
  • Information content filters: These can be used to bury fake news or to manipulate public opinion.

To mitigate these risks, the report recommends pre-publication risk assessments for specific research, selective sharing of sensitive research with trusted organizations, and the establishment of norms within the scientific community regarding dual-use concerns.

Mitigation Strategies and Recommendations

The report provides high-level recommendations for companies, research organizations, practitioners, and governments to ensure a safer AI landscape.

Learning from Cybersecurity Practices

AI researchers should adopt practices from the computer security community to identify and prevent threats. Recommended practices include:

  • Red Teaming: Intentionally attempting to break or subvert systems to find weaknesses.
  • Tech Forecasting: Investing in forecasting to identify threats before they emerge.
  • Confidential Reporting: Establishing conventions for the confidential reporting of vulnerabilities discovered in AI systems.

Broadening the Societal Discussion

Since AI will alter the global threat landscape, the discussion regarding its safety and security must involve a broader cross-section of society, including national security experts, ethicists, civil society, businesses, and the general public.

Concrete Scenarios of AI Misuse

To ground the theoretical risks, the report identifies several concrete scenarios where AI could be misused:

  • Targeted Persuasion: AI-generated persuasive ads targeting the administrator of a security system.
  • Automated Cyberattacks: Cybercriminals using neural networks and "fuzzing" techniques to create computer viruses with automatic exploit generation capabilities.
  • Automated Physical Attacks: Hacking a cleaning robot to deliver an explosives payload to a VIP.
  • Predictive Surveillance: Rogue states using AI-augmented surveillance systems to pre-emptively arrest people based on predictive risk profiles.

Sources