OpenMined/PySyft

Perform data science on data that remains in someone else's server

PySyft – Privacy‑preserving data science platform

What it does
PySyft lets a data owner keep their raw data on their own machine while a data scientist can submit Python jobs that run on that private data. The only thing that leaves the owner’s device are the results that the owner explicitly approves. It works over ordinary cloud‑file services (Google Drive, Microsoft 365) so no special servers are required.

Key concepts

  • Remote Data Science (RDS) client – login_do / login_ds objects represent the data‑owner and data‑scientist roles.
  • Datasets – owners create a mock version (public) and a private version (real data). Scientists can explore the mock data, then submit jobs that automatically run on the private version.
  • Jobs – Python scripts are uploaded, sandboxed in a virtual‑env on the owner’s side, executed, and the approved outputs are synced back.
  • Permissions & peer‑to‑peer auth – each collaborator must be added and approved before any data exchange.
  • Transport‑agnostic, offline‑first – currently implemented over Google Drive, but the sync layer can be swapped; work continues even when peers are offline and syncs later.
  • Enclaves – optional isolated execution environments for extra security (e.g., double‑blind LLM evaluation).

How you start

  1. Install the core library and the RDS client:
    uv pip install "syft>=0.10.0" "syft-rds>=0.6.0"
    Data owners also add syft-bg for background services.
  2. Log in with a Google Cloud OAuth token (Colab does this automatically).
  3. Create a dataset on the owner side, sync, and let the scientist fetch the catalog.
  4. Write a plain Python script (analysis.py) that reads the dataset via syft.resolve_dataset_file_path and writes results.
  5. Submit the script as a job, the owner approves, the job runs in a sandbox, and the scientist retrieves the output.

Main packages

Package Role
syft-rds Remote Data Science client – login, dataset & job APIs
syft-datasets Dataset creation, versioning, mock/private separation
syft-job Job definition, submission, sandboxed execution
syft-permissions / syft-perms Fine‑grained access control
syft-bg Background services, email notifications, TUI dashboard
syft-notebook-ui Helpers for displaying results inside Jupyter

Typical use‑case
A hospital (data owner) stores patient records in a private folder on Google Drive. A researcher (data scientist) writes an analysis script that computes a statistical summary. The script is sent to the hospital, runs inside an isolated virtual environment, and only the approved summary is returned – the raw patient data never leaves the hospital.

Community & support
Developed by the OpenMined foundation, the project is open‑source (Apache 2.0) and has an active Slack community for help. The repo includes unit and integration test suites, and contributors are encouraged to use the just test-* commands for local testing.

All details above are taken directly from the repository’s README.

Related

  • Project
    securefederatedai/openfederatedlearning

    A Python framework for Federated Learning that allows organizations to collaboratively train and validate ML models on sensitive data without sharing the raw data itself.

  • Project
    apple/pfl-research

    A Python framework for simulating private federated learning experiments, supporting multiple ML frameworks and differential privacy mechanisms.

  • Project
    existence-master/Sentient

    A private, local-first personal assistant that manages long-running tasks, remembers user preferences, and integrates with devices and apps without requiring a cloud account.

  • Project
    mostly-ai/mostlyai

    A Python toolkit for creating high-fidelity, privacy-safe synthetic tabular and language data using local or remote compute resources.

  • Project
    NVIDIA/NVFlare

    NVIDIA FLARE is an open-source Python SDK for federated learning and computing that allows multiple sites to collaborate on AI training and data analysis while keeping their raw data local and secure.