Iranian Influence Campaign Uses ChatGPT to Plant Fake Articles in U.S. Media

OpenAI Disrupts Iranian AI-Generated Influence Campaign

OpenAI has identified and banned accounts used by an Iranian influence campaign to plant approximately 100 AI-generated articles across at least 20 U.S. news outlets. The campaign utilized ChatGPT to create polished opinion pieces designed to influence American public opinion, specifically targeting narratives surrounding conflict with Iran.

Campaign Execution and Distribution

The influence operation employed a strategy of submitting AI-generated content to a variety of publications, ranging from small local weeklies to larger platforms with significant social media reach.

• Targeting Small Outlets: One identified piece appeared in the Port St. Joe Star, a small Florida weekly newspaper, warning that the costs of war with Iran would follow Americans beyond the fighting.
• Amplification via Larger Platforms: The campaign used the byline "Ervin Hoskins" to publish critical views of the war in larger publications such as Daily Kos and Middle East Monitor, which collectively maintain a following of over 3 million on Facebook.

OpenAI confirmed that the content attributed to the "Ervin Hoskins" persona was generated using its chatbot by users based in Iran. Following this discovery, OpenAI banned the accounts associated with the campaign.

Technical and Journalistic Implications

The success of this campaign highlights systemic vulnerabilities in the submission and verification processes of various news organizations, particularly those with fewer resources for rigorous fact-checking.

The Role of LLMs in Propaganda

While state-sponsored propaganda and "sock puppet" accounts are established tactics, the integration of Large Language Models (LLMs) like ChatGPT changes the scale and quality of the output. LLMs allow operators to generate high-quality, polished prose rapidly, making fake personas more convincing to editors of smaller publications.

Verification Failures

The ability of an external actor to plant articles in legitimate publications suggests a failure in journalistic verification. The campaign succeeded by creating a professional-looking facade that bypassed the standard scrutiny typically applied to freelance or opinion submissions.

Community Analysis and Perspectives

Discussion among technical communities on Hacker News reveals several critical perspectives on the intersection of AI and disinformation:

Centralization and Oversight

Some observers argue that the centralization of AI tools creates a unique point of failure and surveillance. As one user noted:

"The new thing is a super centralized 'tool' that besides doing your writing for you, logs everything, judges it, and reports to authorities and news outlets as it sees fit."

The Need for Open Weights and Provenance

There is a call for open-weights models to prevent a single entity from controlling the boundaries of acceptable speech, as well as a push for better digital provenance tools. Users suggested that the industry has lacked effective implementation of digital signatures for documents and articles, which could help verify the actual origin of content.

Skepticism of "AI-Driven" Narratives

Some critics argue that the use of AI is a secondary detail in a long history of state-sponsored media manipulation. They suggest that the focus on AI may distract from the fact that intelligence agencies and governments have used similar tactics for decades, regardless of the technology used to draft the text.