OpenAI Disrupts Operation Uncle Spam Influence Activity
OpenAI has banned a series of ChatGPT accounts linked to a China-origin influence operation known as "Operation Uncle Spam." This operation used AI to generate polarized social media content targeting U.S. political discourse to exploit existing divisions rather than promote a single ideology.
AI-Driven Content Generation and Amplification
Operation Uncle Spam utilized OpenAI's models to generate social media comments and calculated amplification tactics. The network primarily focused on divisive topics, such as tariffs, producing content that was either supportive or critical of the issue within the same operational sessions.
To maximize the impact of their messaging, the threat actor requested optimal posting times from the models to increase audience engagement. This indicates a calculated approach to amplifying polarized content across platforms like X and Bluesky.
Persona Creation and Visual Identity
The operation used AI image generation to create stylized logos and profile pictures for fictitious personas. A primary focus was the crafting of personas appearing as U.S. veterans critical of the current U.S. administration, including the creation of logos for entities such as "Veterans for Justice."
Data Extraction and User Analysis
The threat actor consistently requested code and methods to extract personal data from social platforms, including user profiles and follower lists from X and Bluesky. They utilized tools such as Tweepy, Nitter, and the Bluesky public API for these purposes.
Beyond extraction, the accounts used OpenAI's models for guidance on analyzing profile data to identify personal characteristics and organizational affiliations, which potentially allowed the operation to categorize individuals or infer associations and attributes.
Impact Assessment
Using the Breakout Scale for influence operations (IO), OpenAI assesses Operation Uncle Spam as Category 2. This classification indicates activity across multiple platforms with minimal to no breakout or authentic engagement.
Despite some accounts having follower counts in the thousands, the content generated by OpenAI's models received few to no likes or reposts. The follower numbers did not translate into authentic engagement with the posts.