OpenAI Disrupts Korean-language Malware Development Operation
OpenAI has identified and banned a cluster of ChatGPT accounts used by Korean-language operators to engage in malware and command-and-control (C2) development. This action disrupts a coordinated effort to use AI for cyber operations, although OpenAI found no evidence that the models enabled capabilities beyond those already available to the public.
Threat Actor Identification and Attribution
OpenAI observed a cluster of accounts whose activity overlapped with a Trellix report regarding spear phishing campaigns against South Korean diplomatic missions, the deployment of XenoRAT malware, and the use of GitHub-based repositories for C2.
While the observed activity—including the use of the Korean language, operational themes, and activity aligned with UTC+8 and UTC+9 time zones—is consistent with the security community's understanding of North Korean (DPRK) actors, OpenAI stated it cannot independently make an attribution and maintains a block on services in North Korea.
Observed Malicious Behaviors
The operators utilized a structured workflow where multiple accounts were active in narrow time windows, with each account focusing on a specific technical use case rather than spanning multiple areas. Key behaviors included:
Infrastructure and Platform Experimentation: The actors experimented with API scripting and direct-link construction for cloud-storage services including GDrive, pCloud, and file.io, as well as raw content retrieval and token handling via GitHub.
Credential Theft and API Hooking: Interactions with the models focused on Windows API hooking, browser credential and cookie access workflows using DPAPI, and the creation of look-alike verification pages such as reCAPTCHA clones.
Phishing Campaigns: The actors drafted Korean-language phishing emails themed around cryptocurrency, government institutions, and financial service providers.
Technical Capabilities and Model Outputs
Threat actors generated model outputs to support five primary operational areas:
Implant and RAT Development
Operators explored reflective DLL loading, in-memory execution, and Windows API hooking techniques to support Remote Access Trojan (RAT) development.
Credential Theft
Operators generated, modified, and debugged scripts to extract browser encryption keys, cookies, and saved passwords specifically using Chrome and Edge DPAPI workflows.
Phishing and Lures
Actors experimented with HTML obfuscation and the proxying of reCAPTCHA to create convincing login pages, alongside drafting cryptocurrency-themed phishing content.
macOS Development Scaffolding
Requests focused on the development of Safari and Finder extensions and the generation of a sample App Store privacy policy.
Cryptocurrency Operations
Operators used the models to troubleshoot API calls and wallet interactions.
Impact and Mitigation
OpenAI disabled all associated accounts and shared indicators with partners. OpenAI noted that many of the requests fell into a "gray zone of dual-use activity," where legitimate software debugging or cryptography tasks are repurposed for malicious ends. Crucially, the company found no evidence that malicious binaries used in the campaigns described by Trellix were generated using OpenAI's models.