GPT-2 1.5B release notes

OpenAI has released the 1.5 billion parameter version of GPT-2, the final and largest model in its staged release plan. This release includes the model weights and code to facilitate the development of detection tools for synthetic text.

Human Perception of GPT-2 Outputs

Humans find the outputs of the 1.5B model more convincing than smaller versions, though the increase in credibility is marginal compared to the 774M model. In a survey conducted by Cornell University, the 1.5B model received a credibility score of 6.91 out of 10, compared to 6.72 for the 774M model and 6.07 for the 355M model.

Potential for Misuse and Synthetic Propaganda

GPT-2 can be fine-tuned to generate synthetic propaganda for specific ideological positions. Research conducted by the Middlebury Institute of International Studies’ Center on Terrorism, Extremism, and Counterterrorism (CTEC) demonstrated that the model could be fine-tuned on four ideologies: white supremacy, Marxism, jihadist Islamism, and anarchism.

Challenges in Synthetic Text Detection

Content-based detection of synthetic text remains a long-term challenge, as larger models produce outputs that are more difficult to classify. OpenAI developed a detection model based on RoBERTa-BASE (125 million parameters) and RoBERTa-LARGE (355 million parameters) that achieves approximately 95% detection rates for 1.5B GPT-2-generated text.

OpenAI notes that this accuracy is insufficient for standalone detection and must be combined with metadata-based approaches, human judgment, and public education. The company also observed that while training on larger models' outputs improves detection robustness, the overall difficulty of classification increases as model size grows.

Evidence of Misuse

There is currently no strong evidence of widespread misuse of GPT-2. While there has been discussion regarding the potential for high-volume, low-yield operations such as phishing and spam, OpenAI has not seen evidence of the model being used to write code, documentation, or other malicious instances.

Bias Analysis and Standards

Language models inherently contain biases. To address this, OpenAI has implemented two primary strategies:

• Model Cards: Publishing a model card on GitHub to document the inherent issues of the language models.
• Qualitative Evaluation: Performing in-house probes for gender, race, and religious biases to inform the model card.

OpenAI emphasizes that these probes are not comprehensive and highlights the need for the AI research community to collaborate on standardized bias analysis frameworks.

Responsible Publication Norms

The release of GPT-2 served as a test case for a staged release process. OpenAI is continuing to work on establishing responsible publication norms for machine learning through participation in the Partnership on AI’s ‘Responsible Publication Norms for Machine Learning’ project.