OpenAI Disrupts AI-Assisted Task Scam Network

OpenAI has banned a cluster of ChatGPT accounts used by a Cambodia-based network to facilitate "task scams." These scammers used AI to translate communications between Urdu and English to deceive victims into paying "activation fees" for fake high-paying remote jobs.

AI-Powered Social Engineering Workflow

The scam network employed a structured eight-step workflow to move targets from initial engagement to financial loss. The scammers utilized two primary personas: the "recruiter," who handled cold outreach, and the "mentor," who managed the onboarding and psychological manipulation.

1. Initial Recruitment and Engagement

  • Job Posting: Recruiter personas posted ads on online forums for remote work offering high pay for minimal effort, such as $300 per day plus bonuses.
  • Recruitment: Mentor personas contacted potential employees via messaging apps (Telegram, WhatsApp, SMS), claiming to work for luxury design, fashion, and travel brands. The assigned task was to submit five-star reviews to boost customer ratings.

2. Building Trust and Psychological Manipulation

  • Reassurance: When targets expressed skepticism, scammers used AI to generate messages claiming the work was legitimate due to the company's established reputation. In some cases, they claimed to have been scammed themselves in the past to build rapport.
  • Training: Interested targets were moved to Telegram for training. They were given "training accounts" where they performed 25-35 review tasks per day, with the account showing rapidly increasing fake earnings.
  • Excitement: Targets were then offered "special" or "ultimate" review tasks with higher bonuses to increase the perceived value of the potential payout.

3. Financial Extraction and Pressure

  • Activation Fee: Once fake earnings reached a certain threshold, the mentor informed the target they must pay an "activation fee" to withdraw the funds. Scammers urged victims to borrow money from friends or family if necessary.
  • Pressurization: If a target hesitated to pay, the scammer became aggressive, comparing the victim to more efficient workers and threatening the loss of all accumulated earnings.
  • Excuses: After payment, scammers provided various excuses to prevent withdrawals, often demanding further payments to release the funds.

Technical Implementation and Tools

The network primarily used ChatGPT to translate short comments between Urdu and English, facilitating communication between the scammers and their victims. Additionally, the scammers used AI models to proofread the text of websites that spoofed luxury brands to make the scam appear more credible.

Communication Platforms: The operation relied on Telegram as its primary platform, supplemented by WhatsApp, SMS, and various employment forums.

Impact and Detection

OpenAI identified this activity following a tip from Meta. While some generated conversations indicated that targets were highly skeptical—suggesting a low overall conversion rate—online reports and some conversations confirmed that victims did lose money.

"OpenAI’s policies strictly prohibit use of output from our tools for fraud or scams. We are dedicated to collaborating with industry peers and authorities to understand how AI is influencing adversarial behaviors and to actively disrupt scam activities abusing our services."

OpenAI has shared information regarding this disrupted network with industry peers and relevant authorities to enhance collective safety.

Sources