Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models

OpenAI, in collaboration with the Stanford Institute for Human-Centered Artificial Intelligence and other universities, convened a multidisciplinary group of experts to analyze the technical capabilities, limitations, and societal implications of GPT-3. This research effort aimed to identify open questions regarding the largest publicly-disclosed dense language model available at the time of the discussion.

Multidisciplinary Research Framework

The analysis of large language models (LLMs) requires input from diverse academic and professional backgrounds to fully capture their impact. On October 14, 2020, a meeting was held under Chatham House Rules involving experts from the following fields:

  • Computer Science
  • Linguistics
  • Philosophy
  • Political Science
  • Communications
  • Cyber Policy

Core Research Objectives

The discussion was structured around two primary themes to establish a comprehensive understanding of LLM deployment:

Technical Capabilities and Limitations

Researchers focused on defining the boundaries of what large language models can and cannot do. This includes evaluating the performance of GPT-3 across various tasks and identifying the specific technical constraints that limit its reliability or accuracy.

Societal Effects of Widespread Use

The group examined the potential consequences of integrating LLMs into broad societal applications. This involves analyzing how the widespread availability of high-capacity language generation tools affects communication, information integrity, and policy.

Sources