OpenAI Disrupts DPRK-Affiliated Cyber Threat Actors Using AI

OpenAI has banned accounts associated with Democratic People’s Republic of Korea (DPRK)-affiliated threat actors, specifically groups known as VELVET CHOLLIMA (Kimsuky, Emerald Sleet) and STARDUST CHOLLIMA (APT38, Sapphire Sleet). These actors used AI tools to assist in the development of cyber intrusion tools, conduct reconnaissance, and refine social engineering tactics.

Actor Identification and Detection

OpenAI detected these accounts following a tip from a trusted industry partner. The banned accounts demonstrated activity patterns consistent with publicly reported DPRK-affiliated threat actors, specifically VELVET CHOLLIMA and and STARDUST CHOLLIMA.

Behavioral Patterns and AI Usage

The threat actors used OpenAI's models for coding assistance, debugging, and researching open-source security-related code. Their activity focused on two primary areas: cyber intrusion operations and cryptocurrency-related topics, a combination typical of DPRK-associated groups.

Technical AI-Assisted Activities

The actors leveraged LLMs for the following specific technical tasks:

  • LLM-Informed Reconnaissance: Asking about vulnerabilities in various applications.
  • LLM-Aided Development: Developing and troubleshooting a C#-based RDP client for brute-force attacks and requesting code to bypass security warnings for unauthorized RDP access.
  • LLM-Enhanced Scripting: Requesting PowerShell scripts for RDP connections, file upload/download, executing code from memory, and obfuscating HTML content to evade anomaly detection.
  • LLM-Optimized Payload Crafting: Discussing the creation and deployment of obfuscated payloads for execution.
  • LLM-Assisted Post-Compromise Activity: Researching open-source Remote Administration Tools (RATs).
  • LLM-Supported Social Engineering: Seeking methods for targeted phishing against cryptocurrency investors and traders, as well as crafting phishing emails to manipulate users into revealing sensitive information.

Security Impact and Mitigation

OpenAI reports that the model generations provided to the actors did not offer any novel capabilities, as the prompts and queries were primarily based on existing open-source information. In many cases, the model refused to respond to the requests.

To disrupt these operations, OpenAI banned the associated accounts and shared the discovered payloads with the security community. During the investigation into the actor's attempts to debug auto-start extensibility point (ASEP) locations and techniques for MacOS, OpenAI identified staging URLs for binaries that were previously unknown to security vendors. By submitting these URLs to online scanning services, OpenAI enabled security vendors to now reliably detect these binaries, providing protection for potential victims.

Sources