OpenAI Operation ScopeCreep: Disrupting Russian-speaking Malware Development

OpenAI has banned a cluster of ChatGPT accounts used by a Russian-speaking threat actor to develop and refine Windows malware, debug code, and establish command-and-control (C2) infrastructure. This operation, dubbed "ScopeCreep," demonstrates how threat actors attempt to use large language models (LLMs) to accelerate malware development cycles while employing strict operational security to evade detection.

Malware Distribution and Execution

ScopeCreep distributed its malware by impersonating a popular gaming tool called "Crosshair-X" through a public code repository. Users who downloaded the malicious version triggered a loader that downloaded and executed additional malicious files from the attacker's infrastructure.

Once executed, the malware followed a multi-stage process designed to achieve the following:

  • Privilege Escalation: Increasing system permissions to gain deeper access.
  • Persistence: Establishing stealthy mechanisms to remain on the system.
  • Exfiltration: Stealing sensitive data and notifying the attacker via Telegram.
  • Evasion: Avoiding detection through various technical means.

Technical Capabilities and Evasion Techniques

While OpenAI notes that the capabilities of ScopeCreep are not particularly novel, the malware employed a range of sophisticated evasion and execution techniques:

  • DLL Side-loading: Used for stealthy execution.
  • Custom Packing: Utilizing Themida for obfuscation.
  • C2 Payloads: Designed to avoid signature-based detection.
  • Network Obfuscation: Using HTTPS over port 80 and proxy-based traffic obfuscation.
  • Credential Theft: Targeting session and credential data.

LLM-Assisted Development Patterns

The threat actor utilized ChatGPT iteratively, requesting specific feature implementations and debugging assistance. To maintain operational security, the actor used temporary email addresses to create new accounts for every single incremental improvement to the code, abandoning accounts after a single conversation.

OpenAI mapped these activities to the LLM ATT&CK framework, identifying five key areas of model-assisted development:

  • LLM Aided Development: Compiling python310.dll to ensure code execution whenever python.exe runs; troubleshooting SSL/TLS certificates for HTTPS traffic on port 80; and migrating a Flask-based C2 server to a production-ready WSGI server.
  • LLM-Enhanced Anomaly Detection Evasion: Developing PowerShell commands within Go to programmatically add antivirus (AV) exclusions to Windows Defender settings. -- LLM-Assisted Post-Compromise Activity: Debugging code used to notify an attacker-controlled Telegram channel upon the compromise of a new victim.

Detection and Mitigation

OpenAI detected the activity through its scaled cyber abuse detection process. The threat was mitigated through coordination with the code hosting provider to remove the malicious repository and the banning of all associated ChatGPT accounts.

While samples of the malware appeared on VirusTotal, OpenAI reports no evidence of widespread distribution or interest. The actor's attempt to use LLMs to speed up development provided a paradoxical opportunity for OpenAI to identify and disrupt the the operation in its early stages.

Sources