Disparate Vulnerability to Membership Inference Attacks (2019-06-02T00:00:00.000000Z)

TL;DR

This paper establishes necessary and sufficient conditions for MIAs to be prevented, both on average and for population subgroups, and derives connections of disparate vulnerability to algorithmic fairness and to differential privacy, and establishes which attacks are suitable for estimating disparate vulnerability.

Abstract

Abstract A membership inference attack (MIA) against a machine-learning model enables an attacker to determine whether a given data record was part of the model’s training data or not. In this paper, we provide an in-depth study of the phenomenon of disparate vulnerability against MIAs: unequal success rate of MIAs against different population subgroups. We first establish necessary and sufficient conditions for MIAs to be prevented, both on average and for population subgroups, using a notion of distributional generalization. Second, we derive connections of disparate vulnerability to algorithmic fairness and to differential privacy. We show that fairness can only prevent disparate vulnerability against limited classes of adversaries. Differential privacy bounds disparate vulnerability but can significantly reduce the accuracy of the model. We show that estimating disparate vulnerability by naïvely applying existing attacks can lead to overestimation. We then establish which attacks are suitable for estimating disparate vulnerability, and provide a statistical framework for doing so reliably. We conduct experiments on synthetic and real-world data finding significant evidence of disparate vulnerability in realistic settings.

Authors

Bogdan Kulynych

2 papers

Mohammad Yaghini

1 papers

Giovanni Cherubin

2 papers

TL;DR

Abstract

Authors

References48 items

The (Im)possibility of fairness

On the Privacy Risks of Algorithmic Fairness

Bayes Security: A Not So Average Metric

Differentially Private Learning Does Not Bound Membership Inference

Distributional Generalization: A New Kind of Generalization

A Pragmatic Approach to Membership Inferences on Machine Learning Models

Revisiting Membership Inference Under Realistic Assumptions

Fairlearn: A toolkit for assessing and improving fairness in AI

Systematic Evaluation of Privacy Risks of Machine Learning Models

Membership Inference Attacks and Defenses in Classification Models

2020 Ends a Decade of 62 New Data Privacy Laws

Modelling and Quantifying Membership Information Leakage in Machine Learning

Diffprivlib: The IBM Differential Privacy Library

On the Privacy Risks of Model Explanations

Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference

Inherent Tradeoffs in Learning Fair Representation

Fair decision making using privacy-protected data

Differential Privacy Has Disparate Impact on Model Accuracy

White-box vs Black-box: Bayes Optimal Strategies for Membership Inference

F-BLEAU: Fast Black-Box Leakage Estimation

Comprehensive Privacy Analysis of Deep Learning: Stand-alone and Federated Learning under Passive and Active White-box Inference Attacks

The Frontiers of Fairness in Machine Learning

Algorithms that remember: model inversion attacks and data protection law

ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

Privacy for All: Ensuring Fair and Equitable Privacy Protections

Does mitigating ML's impact disparity require treatment disparity?

Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting

Fairness in Criminal Justice Risk Assessments: The State of the Art

Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments

Membership Inference Attacks Against Machine Learning Models

Equality of Opportunity in Supervised Learning

Predicting the Future - Big Data, Machine Learning, and Clinical Medicine.

Fairness through awareness

Scikit-learn: Machine Learning in Python

Consumer Credit Risk Models Via Machine-Learning Algorithms

Differentially Private Empirical Risk Minimization

Differential Privacy

Experimental Design and Analysis

Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid

On the learnability of discrete distributions

The Localisation Gambit: Unpacking Policy Measures for Sovereign Control of Data in India

Big Data's Disparate Impact

To predict and serve? Significance

Cynthia Dwork

A Probabilistic Theory of Pattern Recognition

A Proofs In this section we provide the omitted proofs

Proposition 1. V ( A ∗ W,Z ) ≥ V ( A ∗ W )

Disparate Vulnerability to Membership Inference

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names