meta-pytorch/opacus
Training PyTorch models with differential privacy
What it solves
Opacus allows developers to train PyTorch models with differential privacy, ensuring that the training process does not leak sensitive information about individual data points in the training set.
How it works
It provides a PrivacyEngine that wraps existing PyTorch models, optimizers, and data loaders. By using the make_private() method, the library implements differentially private stochastic gradient descent (DP-SGD) and allows users to track the privacy budget expended during training in real-time.
Who it’s for
- ML practitioners who want a simple way to integrate differential privacy into their models with minimal code changes.
- Differential Privacy researchers who need a flexible tool for experimenting and tinkering with privacy-preserving ML algorithms.
Highlights
- Minimal integration effort: Requires very few changes to existing PyTorch training loops.
- Privacy budget tracking: Enables online tracking of the privacy loss during training.
- Memory efficiency: Supports Fast Gradient Clipping and Ghost Clipping to reduce the memory overhead of DP-SGD.
- Broad compatibility: Works with various architectures, including LSTMs and BERT (via LoRA and peft).