OpenAI Improving Language Understanding with Unsupervised Learning

OpenAI has developed a two-stage training process that enables a single Transformer model to achieve high performance across a wide variety of natural language processing (NLP) tasks. By first training a model on a large volume of data using unsupervised language modeling and then fine-tuning it on smaller, task-specific supervised datasets, the system can solve complex problems with minimal adaptation.

Two-Stage Training Architecture

The system utilizes a pipeline consisting of unsupervised pre-training followed by supervised fine-tuning. This approach allows the model to develop a general understanding of language before being applied to specific tasks.

• Unsupervised Pre-training: The model is a Transformer trained on a very large amount of data using language modeling as the training signal. This stage allows the model to learn discriminative features from raw text without manual labeling.

• Supervised Fine-tuning: The pre-trained core model is then adapted to specific tasks using much smaller supervised datasets. This process requires minimal adaptation to transition the model from a general language model to a task-specific solver.

Comparison to Prior Research

This work builds upon and extends several previous methodologies in unsupervised and semi-supervised learning:

• Semi-supervised Sequence Learning: This research established the method of using unsupervised pre-training of an LSTM followed by supervised fine-tuning to improve document classification.
• ULMFiT: This work demonstrated that a single dataset-agnostic LSTM language model could be fine-tuned for state-of-the-art performance on various document classification datasets.
• ELMo: While ELMo also incorporates pre-training, OpenAI's approach is more task-agnostic, whereas ELMo uses task-customized architectures to achieve its results.

Performance and Capabilities

The model demonstrates strong performance across a broad range of tasks beyond simple document classification, including semantic similarity, reading comprehension, and commonsense reasoning.

Commonsense Reasoning and World Knowledge

One of the most significant results is the model's performance on the COPA, RACE, and ROCStories datasets. The model achieved new state-of-the-art results on these datasets by a wide margin. Because these tasks are thought to require multi-sentence reasoning and significant world knowledge, the results suggest that the model improves these capabilities predominantly through the unsupervised learning stage.

Task Versatility and Minimal Tuning

The system is designed for efficiency and consistency. All datasets were processed using a single forward language model without ensembling, and the majority of the reported results were achieved using the exact same hyperparameter settings, indicating that very little tuning was required to achieve these results.

Evaluated Datasets and Task Examples

The model was tested across a diverse set of NLP benchmarks, including:

Dataset	Task Type	Example
SNLI	Natural Language Inference	Determining if two sentences contradict (Contra.) or entail each other.
SciTail	Natural Language Inference	Determining if a sentence entails or is neutral relative to another.
QNLI	Natural Language Inference	Determining if a statement entails a context.
RTE	Natural Language Inference	Determining if a sentence contradicts another.
STS-B	Semantic Textual Similarity	Scoring the similarity between two sentences (e.g., 2/5).
QQP	Quora Question Pairs	Determining if two questions are the same.
MRPC	Microsoft Research Paraphrase Corpus	Determining if two sentences are the same.
RACE	Reading Comprehension	Answering multiple-choice questions based on a text passage.
ROCStories	Commonsense Reasoning	Selecting the most likely story conclusion.
COPA	Commonsense Reasoning	Identifying the cause or effect of an event.
SST-2	Sentiment Analysis	Classifying text as positive or negative.
CoLA	Linguistic Acceptability	Determining if a sentence is grammatically acceptable.