meta-pytorch/opacus

Training PyTorch models with differential privacy

What it solves

Opacus 允许开发者使用差分隐私训练 PyTorch 模型，确保训练过程不会泄露训练集中单个数据点的敏感信息。

How it works

它提供了一个 PrivacyEngine，可以封装现有的 PyTorch 模型、优化器和数据加载器。通过使用 make_private() 方法，该库实现了差分隐私随机梯度下降 (DP-SGD)，并允许用户在训练期间实时跟踪隐私预算的消耗。

Who it’s for

  • ML practitioners 想要一种简单的方式，以最少的代码更改将差分隐私集成到他们的模型中。
  • Differential Privacy researchers 需要一个灵活的工具，用于实验和调整隐私保护机器学习算法。

Highlights

  • Minimal integration effort: Requires very few changes to existing PyTorch training loops.
  • Privacy budget tracking: Enables online tracking of the privacy loss during training.
  • Memory efficiency: Supports Fast Gradient Clipping and Ghost Clipping to reduce the memory overhead of DP-SGD.
  • Broad compatibility: Works with various architectures, including LSTMs and BERT (via LoRA and peft).