Privacy Preserving AI (Andrew Trask) | MIT Deep Learning Series

Core Problem and Motivation

• Data Scarcity Barrier: High-quality, private datasets (e.g., medical records for tumors, dementia, or diabetes) are difficult to acquire, highly regulated, and expensive, forcing researchers to rely on accessible but less impactful datasets like ImageNet or MNIST.
• Current Limitation: The machine learning community spends the majority of resources on tasks with open data because accessing sensitive private data requires complex business models, legal negotiations, or institutional partnerships.
• Goal: To lower the barrier to entry for privacy-preserving AI by enabling researchers to "pip install" access to private datasets without physically seeing the raw data.
• OpenMind Community: A 6,000-person open-source community led by Andrew Trask focused on developing tools (like PySyft) to make algorithms and data more privacy-preserving.

Technical Tools and Mechanisms

• Remote Execution (Torch Hook): Extends deep learning frameworks (e.g., PyTorch) to allow computation on remote workers (e.g., hospital data centers) without direct data access.
  • Workflow: A tensor is sent to a remote worker via .send(), returning a pointer; computations run locally in code but execute remotely on the data, returning new pointers to results.
  • Control: Access to raw data is restricted via .get() requests, which require specific permissions to prevent unauthorized data exfiltration.
• Remote Search and Sampling: Enables data scientists to query metadata and access sample data (e.g., GAN-generated samples or short snippets) to perform feature engineering without seeing the full dataset.
  • Utility: Allows for remote normalization and quality evaluation by exposing structural details (column types, value ranges) without revealing individual records.
• Differential Privacy (DP): Adds mathematical noise to queries or data to provide a formal privacy budget and prevent linkage attacks.
  • Mechanism: Ensures that removing or replacing one individual in the database does not significantly alter the query output.
  • Privacy Budget ($\epsilon$): A strict limit on statistical uniqueness allowed to be published; budgets sum across queries, requiring a total budget cap per project or individual.
  • Local vs. Global DP: Local DP adds noise before data leaves the source (highest protection), while Global DP adds noise to the query output after processing (better utility but requires trusting the data owner).
  • Anonymization Failure: Simple redaction (e.g., Netflix Prize) is ineffective because statistical uniqueness allows re-identification via linkage with external datasets (e.g., IMDb or voter records).
• Secure Multi-Party Computation (MPC): Allows multiple parties to compute functions on shared data without revealing individual inputs to each other.
  • Secret Sharing: Data is split into encrypted shares (e.g., 2 and 3 for a total of 5) distributed among participants; no single share reveals the original number.
  • Computation: Mathematical operations (including multiplication) can be performed on encrypted shares, with the result remaining encrypted until all parties agree to decrypt.
  • Security Models: Supports "active security" to detect unauthorized computation attempts and "passive security" for standard confidentiality.

Limitations and Challenges

• Computational Overhead: Encrypted computation (MPC) introduces significant latency; current state-of-the-art deep learning prediction is approximately 13x slower than plain text execution.
• Model Security Risks: Sending models to remote sites exposes intellectual property (e.g., a $10M healthcare model) to potential theft or tampering.
• Cross-Owner Joins: Joining data across multiple untrusted owners remains difficult due to the lack of a unified trust framework.
• Non-Linear Functions: Performance hits are most severe for non-linear operations (e.g., ReLUs); research is currently focused on polynomial approximations and discrete comparison functions to mitigate this.
• Federated Learning Distinction: While Google's federated learning trains models on local devices, it does not inherently prevent data memorization and requires integration with differential privacy for strong guarantees.

Societal and Economic Vision

• End-to-End Encrypted Services: A framework combining MPC (input privacy), logic execution, and DP (output privacy) to allow services (e.g., medical diagnosis) to operate without the provider ever seeing user data or the raw inputs.
  • Example: A user can receive a skin cancer prediction from a hospital's model without revealing their image or receiving a diagnosis that could be linked back to them by the service provider.
• Single-Use Accountability: Systems designed to answer specific questions (e.g., "Is there a weapon in this bag?") without revealing the entire dataset, analogous to a sniffer dog, reducing information leakage in surveillance and enforcement.
• Open Data for Science: Unlocking the "untapped" data already held by enterprises, governments, and hospitals to create massive new datasets for training AI, potentially driving breakthroughs comparable to the introduction of GPUs or ImageNet.
• Commercial Viability: Protecting data allows owners to maintain scarcity and value, shifting the business model from selling data copies to selling "data access" or query services.
• Infrastructure Needs: Future adoption requires enterprise-level implementation first, followed by the development of a "privacy budget accounting system" (possibly decentralized or via data banks) to track an individual's total $\epsilon$ spending across multiple institutions.

Future Outlook and Q&A Insights

• Regulatory Adoption: Differential privacy is gaining traction in government (e.g., US Census 2020), though specific mandates for medical algorithms (FDA) are not yet standardized.
• Bias Mitigation: Debiasing encrypted models is possible by provisioning sufficient privacy budget to allow for introspection and weight adjustment without exposing raw data.
• Recommendation Systems: Current systems lack holistic targets (e.g., sleep quality, life satisfaction) due to data inaccessibility; privacy-preserving AI could enable recommenders to optimize for well-being using private user data without exposure.
• Implementation Timeline: The technology is theoretically mature; the immediate goal is pilot programs in the current year, with broader individual privacy budget control dependent on long-term infrastructure development.