OpenAI Semi-supervised Knowledge Transfer for Private Training Data
OpenAI has introduced Private Aggregation of Teacher Ensembles (PATE), a framework designed to protect sensitive training data, such as medical histories, from being leaked through a machine learning model. PATE ensures that a model cannot inadvertently store and reveal sensitive information by decoupling the training data from the the final published model through a teacher-student architecture.
Private Aggregation of Teacher Ensembles (PATE) Architecture
PATE utilizes a black-box approach to transfer knowledge from multiple teacher models to a student model. The process works as follows:
- Teacher Training: Multiple models (teachers) are trained on disjoint subsets of sensitive data. Each teacher only has access to a specific subset of users or records.
- Noisy Voting: To generate labels for a student model, the teachers vote on the output for a given input. Noise is added to this voting process to provide formal privacy guarantees.
- Student Learning: A student model is trained to predict the output of this noisy vote. The student model never has direct access to the individual teacher models, the underlying sensitive training data, or the teacher's internal parameters.
This architecture prevents any single teacher—and therefore any single dataset—from dominating the student's training, ensuring that the specific details of any individual record are not memorized by the final model.
Privacy Guarantees and Differential Privacy
PATE provides strong privacy guarantees that hold even if an adversary can inspect the internal workings of the student model. These guarantees are formally grounded in the framework of differential privacy, which ensures that the presence or absence of any single individual's data in the training set does not significantly alter the output of the student model.
Technical Advantages and Versatility
Unlike previous privacy-preserving machine learning methods, PATE imposes only weak assumptions on how teachers are trained. This allows the framework to be applicable to any model type, including non-convex models such as Deep Neural Networks (DNNs).
By combining an improved privacy analysis with semi-supervised learning, the researchers achieved state-of-the-art privacy/utility trade-offs on the MNIST and SVHN datasets.
Summary of Privacy Implications
| Feature | PATE Approach |
|---|---|
| Data Access | Student model has no direct access to sensitive data |
| Mechanism | Noisy voting among disjoint teacher ensembles |
| Privacy Framework | Differential Privacy |
| Model Compatibility | Compatible with any model, including DNNs |