OpenAI Interpretable Machine Learning through Teaching
OpenAI has developed a machine teaching approach that enables computers and humans to collaborate by creating a human-interpretable language for concept recognition. This method uses a cooperative game between a teacher network and a student network to identify the smallest set of illustrative examples that allow a student to correctly guess a target concept.
The Teacher-Student Framework
The core of this approach is a cooperative game where a teacher network selects examples to help a student network guess a specific concept (such as "dog" or "zebra"). The goal is for the teacher to learn to select the most illustrative examples possible to facilitate the student's understanding.
Two-Stage Training Process
To ensure the resulting examples remain interpretable to humans, OpenAI uses a two-stage technique:
- Student Training: A student neural network is first trained using traditional supervised learning methods on randomly selected input examples to guess correct concept labels.
- Teacher Optimization: A teacher network, which has access to labels linking concepts to examples, tests various examples on the student. The teacher converges on the smallest set of examples required for the student to correctly identify the intended concept.
Avoiding Collusion and Arbitrary Encoding
By separating the training stages, this method prevents the two networks from developing a private, non-human-interpretable language. In joint training—where student and teacher are trained simultaneously—the networks often collude to communicate via arbitrary vectors or examples that make no sense to humans. For example, a concept like "dog" could be encoded through images of llamas and motorcycles, or a rectangle's dimensions could be encoded by two random-looking dots.
Practical Application: Concept Recognition
To demonstrate the effectiveness of the technique, OpenAI researchers used examples that varied based on four properties: size (small, medium, large), color (red, blue, green), shape (square vs circle), and border (solid vs none).
In this instance, a concept is defined as a set of properties that define a subset of examples. The teacher network learns to pick examples that share only the properties required by the the concept, allowing the student to rule out irrelevant properties.
Example: Teaching the concept of "red"
To teach the student to recognize the concept of "red", the teacher selects two specific examples: a large red square with no border and a small red circle with a border. Because the only common property between these two shapes is the color red, the student can logically conclude that the concept is defined by the color red.