Measuring Progress on Scalable Oversight for Large Language Models

Anthropic has introduced a framework for measuring progress on scalable oversight, which is the challenge of supervising AI systems that possess skills exceeding human capabilities. This research is critical because developing safe and useful general-purpose AI requires a method for humans to oversee systems that may outperform them on most task-relevant skills.

The Challenge of Scalable Oversight

Scalable oversight refers to the problem of supervising AI systems that potentially outperform humans on most skills relevant to a specific task. The primary difficulty in studying this empirically is that current general AI systems do not yet broadly exceed human abilities across all domains.

Experimental Design for Scalable Oversight

To study scalable oversight with present-day models, Anthropic proposes an experimental design centered on tasks where human specialists succeed, but both unaided humans and current general AI systems fail. This approach allows researchers to use human specialists as a ground truth for correctness, while testing if a model can assist a non-specialist human in reaching that specialist level of performance.

Proof-of-Concept Results

Anthropic conducted a proof-of-concept experiment using two question-answering tasks: MMLU (Massive Multitask Language Understanding) and time-limited QuALITY. The experiment tested a baseline strategy for scalable oversight where human participants interacted with an unreliable large-language-model (LLM) dialog assistant via chat.

Key findings from this experiment include:

• Improved Human Performance: Human participants interacting with the LLM assistant substantially outperformed their own unaided performance.
• Improved System Performance: The human-AI collaboration outperformed the model acting alone.
• Viability of Study: The results indicate that scalable oversight is tractable to study using current models, even when those models are unreliable.

Implications for AI Safety

These findings bolster recent evidence that large language models can productively assist humans with difficult tasks. By demonstrating that humans can use LLMs to improve their own performance on complex tasks, the researchers suggest that a path forward exists for humans to maintain oversight of AI systems as their capabilities increase.