Creating Privacy Preserving AI with Substra

Federated Learning Enables Privacy-Preserving AI Training

Federated learning (FL) is a decentralized machine learning technique that allows models to be trained across multiple data providers without requiring the data to be centralized on a single server. Instead of moving raw data, only the resulting model weights are transferred between servers, ensuring that sensitive information remains on local servers.

This approach is critical for domains such as healthcare, where data is often siloed across academic centers and medical institutions. Regulations like HIPAA protect patient privacy, which can limit the volume of data available to data scientists. Federated learning works alongside these regulations to unlock these silos, providing the volume of data necessary to train impactful models while maintaining strict security and privacy.

Benefits of Decentralized Training

Training models through federated learning provides several advantages over single-source training:

  • Increased Robustness and Representation: By utilizing data from different sources, models become more representative and robust.
  • Reduced Bias: FL reduces the risk of bias caused by variations in underlying datasets, such as differences in demographic distributions of patient populations or variations in data capture equipment and techniques.
  • Improved Generalization: Models trained on multiple data sources typically perform better in real-world settings than those trained on a single source.

Substra: An Open Source Framework for Production FL

Substra is an open-source federated learning framework designed for real-world production environments. While federated learning has progressed significantly over the last decade, Substra focuses on the practical deployment and architecture of federated networks in complex security environments and IT infrastructures.

Real-World Applications and Impact

Substra has been utilized to enable significant research milestones, including:

  • Drug Discovery: In the MELLODDY project, 10 competing biopharma companies collaborated by sharing the world's largest collection of small molecules with known biochemical or cellular activity to build more accurate predictive models.
  • Medical Research: The framework has enabled breakthroughs in breast cancer research.

Integration with Hugging Face

Hugging Face collaborated with Substra to create a dedicated Space to demonstrate the real-world challenges of AI research, specifically the lack of centralized, high-quality data. The Space allows users to control the distribution of samples and observe how a model trained with federated learning consistently outperforms models trained on data from a single source when tested on validation data.

Complementary Privacy Enhancing Technologies (PETs)

Federated learning is part of a broader ecosystem of privacy-enhancing technologies (PETs). To create multi-layered privacy-preserving environments, FL can be combined with other technologies such as:

  • Secure Enclaves
  • Multi-party Computation

Sources