OpenAI Disrupts Operation Stop News Russian Influence Activity

OpenAI has banned a cluster of ChatGPT accounts associated with a Russia-origin influence operation known as "Stop News." This operation utilized AI-generated content to mimic news outlets and social media accounts across multiple platforms to spread specific narratives and promotional materials.

Operation Overview and Actor Profile

Operation Stop News originated in Russia and appeared to be run by a marketing company. OpenAI initiated its investigation following off-platform indicators found in Meta’s 2024 second quarter threat report. The operation's primary goal was to generate content in English, French, and Russian for distribution across various websites and social media platforms.

AI-Driven Content Generation and Distribution

Stop News used OpenAI models to generate a diverse range of content, including short comments, long-form articles, and imagery. The distribution strategy varied by language and target region:

  • English and French: Long-form articles were posted on websites posing as news outlets in the UK and Africa. Short comments were distributed via X (formerly Twitter), and potentially Facebook and Instagram.

  • Russian: Content served as promotional materials on Russian-language forums and websites, including responses to customer feedback for companies in the St. Petersburg region.

Use of Visuals and Imagery

The operation was notably prolific in its use of DALL·E for image generation. These images typically featured cartoon styles, bright colors, or dramatic tones designed to attract attention and increase engagement. OpenAI noted that these images were not intended as deepfakes or to falsely represent events, but rather to make posts more eye-catching.

Additionally, the operation mixed modern AI with older technology. While DALL·E was used for content, profile pictures for social media accounts were often created using Generative Adversarial Networks (GANs), a technique common in influence operations since 2019.

Content Themes and Narratives

The content generated by Stop News focused on different themes depending on the target audience:

  • United Kingdom: Content on britishtalks[.]com focused on non-political news, sports, and culture. However, euronewstop[.]co[.]uk mingled this content with criticism of Ukraine and the UK’s support for Ukraine.

  • Africa (French language): Articles and comments focused on West African music and culture, interspersed with criticism of Ukraine and France, including accusations that France failed in its organization of the Paris Olympic Games.

  • Russia: Content primarily promoted commercial services in north-western Russia, specifically the St. Petersburg region.

Impact and Engagement Analysis

Despite the use of AI, the operation's social media engagement remained low. Most X accounts had double-digit follower counts, and Meta reported a total following of 2,100 across Facebook and Instagram.

However, the operation achieved a unique level of success in establishing "information partnerships" with legitimate local organizations. These partners included a school in Wales, a church in Yorkshire, and an association of chambers of commerce in California. These organizations published information about these partnerships on their own websites, allowing the operation's "news" brands to engage with their members through joint publications and community outreach.

Breakout Scale Assessment

OpenAI assesses this operation as Category Three on the Breakout Scale (a scale of 1 to 6). This rating is based on the following:

  • Activity: Posting activity occurred across multiple platforms.

  • Engagement: There was evidence that real people, specifically the "information partners," engaged with the operation, but it lacked the virality of larger-scale Russian "troll" operations.

Associated Domains

OpenAI identified the following domains associated with Operation Stop News:

  • Euronewstop[.]co[.]uk
  • Newstop[.]africa
  • Britishtalks[.]com
  • Britishattitudes[.]com

Sources