Membership Inference Attacks on Machine Learning: A Survey (2021-03-14T00:00:00.000000Z)

TL;DR

This article provides the taxonomies for both attacks and defenses, based on their characterizations, and discusses their pros and cons, and point out several promising future research directions to inspire the researchers who wish to follow this area.

Abstract

Machine learning (ML) models have been widely applied to various applications, including image classification, text generation, audio recognition, and graph data analysis. However, recent studies have shown that ML models are vulnerable to membership inference attacks (MIAs), which aim to infer whether a data record was used to train a target model or not. MIAs on ML models can directly lead to a privacy breach. For example, via identifying the fact that a clinical record that has been used to train a model associated with a certain disease, an attacker can infer that the owner of the clinical record has the disease with a high chance. In recent years, MIAs have been shown to be effective on various ML models, e.g., classification models and generative models. Meanwhile, many defense methods have been proposed to mitigate MIAs. Although MIAs on ML models form a newly emerging and rapidly growing research area, there has been no systematic survey on this topic yet. In this article, we conduct the first comprehensive survey on membership inference attacks and defenses. We provide the taxonomies for both attacks and defenses, based on their characterizations, and discuss their pros and cons. Based on the limitations and gaps identified in this survey, we point out several promising future research directions to inspire the researchers who wish to follow this area. This survey not only serves as a reference for the research community but also provides a clear description for researchers outside this research domain. To further help the researchers, we have created an online resource repository, which we will keep updated with future relevant work. Interested readers can find the repository at https://github.com/HongshengHu/membership-inference-machine-learning-literature.

Authors

References268 items

Confusion Matrix

Practical Membership Inference Attack Against Collaborative Inference in Industrial IoT

Generative Adversarial Networks

Adapting Membership Inference Attacks to GNN for Graph Classification: Approaches and Implications

Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture

Foundations of Machine Learning

Digestive neural networks: A novel defense strategy against inference attacks in federated learning

Membership Inference Attacks Against Recommender Systems

Source Inference Attacks in Federated Learning

Evaluating the Vulnerability of End-to-End Automatic Speech Recognition Models to Membership Inference Attacks

EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learning

PAR-GAN: Improving the Generalization of Generative Adversarial Networks Against Membership Inference Attacks

Defending Privacy Against More Knowledgeable Membership Inference Attackers

Membership Inference Attacks on Lottery Ticket Networks

EAR: An Enhanced Adversarial Regularization Approach against Membership Inference Attacks

This Person (Probably) Exists. Identity Membership Attacks Against GAN Generated Faces

Trustworthy AI: A Computational Perspective

A Comprehensive Survey of Privacy-preserving Federated Learning

Deep learning for AI

Membership Inference on Word Embedding and Beyond

A Survey of Unsupervised Generative Models for Exploratory Data Analysis and Representation Learning

On the Difficulty of Membership Inference Attacks

Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain

Membership Privacy for Machine Learning Models Through Knowledge Transfer

Membership Inference Attacks on Deep Regression Models for Neuroimaging

privGAN: Protecting GANs from membership inference attacks at low cost to utility

Membership Inference Attack Susceptibility of Clinical Language Models

Membership Inference Attacks on Knowledge Graphs

On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models

On the privacy-utility trade-off in differentially private hierarchical text classification

Defending Medical Image Diagnostics against Privacy Attacks using Generative Methods

A Taxonomy of Attacks on Federated Learning

Understanding deep learning (still) requires rethinking generalization

Node-Level Membership Inference Attacks Against Graph Neural Networks

ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models

Membership Inference Attack on Graph Neural Networks

Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning

Practical Blind Membership Inference Attack via Differential Comparisons

Membership Inference Attack with Multi-Grade Service Models in Edge Intelligence

Extracting Training Data from Large Language Models

When Machine Learning Meets Privacy

Privacy-Preserving in Defending against Membership Inference Attacks

On the Privacy Risks of Algorithmic Fairness

FaceLeaks: Inference Attacks against Transfer Learning Models via Black-box Queries

Differentially Private Learning Does Not Bound Membership Inference

Quantifying Membership Privacy via Information Leakage

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

Quantifying Privacy Leakage in Graph Embedding

GECKO: Reconciling Privacy, Accuracy and Efficiency in Embedded Deep Learning

An Extension of Fano's Inequality for Characterizing Model Susceptibility to Membership Inference Attacks

Quantifying Membership Inference Vulnerability via Generalization Gap and Other Model Metrics

Privacy Analysis of Deep Learning in the Wild: Membership Inference Attacks against Transfer Learning

Toward Robustness and Privacy in Federated Learning: Experimenting with Local and Central Differential Privacy

Investigating the Impact of Pre-trained Word Embeddings on Memorization in Neural Networks

A Comprehensive Analysis of Information Leakage in Deep Transfer Learning

Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries

A Pragmatic Approach to Membership Inferences on Machine Learning Models

Against Membership Inference Attack: Pruning is All You Need

Differential Privacy Protection Against Membership Inference Attack on Machine Learning for Genomic Data

Beyond Model-Level Membership Privacy Leakage: an Adversarial Approach in Federated Learning

Membership Leakage in Label-Only Exposures

Label-Leaks: Membership Inference Attack with Label

Label-Only Membership Inference Attacks

How Does Data Augmentation Affect Privacy in Machine Learning?

ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning

A Survey of Privacy Attacks in Machine Learning

Auditing Differentially Private Machine Learning: How Private is Private SGD?

Adversarial Examples on Object Recognition

On the Effectiveness of Regularization Against Membership Inference Attacks

GAN Enhanced Membership Inference: A Passive Local Attack in Federated Learning

Revisiting Membership Inference Under Realistic Assumptions

An Overview of Privacy in Machine Learning

A Secure Federated Learning Framework for 5G Networks

Defending Model Inversion and Membership Inference Attacks via Prediction Purification

When Machine Unlearning Jeopardizes Privacy

Diabetic Retinopathy Detection

Privacy in Deep Learning: A Survey

Meta-Learning in Neural Networks: A Survey

Racism and discrimination in COVID-19 responses

Information Leakage in Embedding Models

Systematic Evaluation of Privacy Risks of Machine Learning Models

Improved Baselines with Momentum Contrastive Learning

Threats to Federated Learning: A Survey

Membership Inference Attacks and Defenses in Classification Models

Membership Inference Attacks and Defenses in Supervised Learning via Generalization Gap

Data and Model Dependencies of Membership Inference Attack

Modelling and Quantifying Membership Information Leakage in Machine Learning

Privacy for All: Demystify Vulnerability Disparity of Differential Privacy against Membership Inference Attack

privGAN: Protecting GANs from membership inference attacks at low cost

Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer

Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentation

Federated Learning

Machine Unlearning

Effects of Differential Privacy and Data Skewness on Membership Inference Vulnerability

Momentum Contrast for Unsupervised Visual Representation Learning

Demystifying the Membership Inference Attack

A taxonomy and terminology of adversarial machine learning

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

FedMD: Heterogenous Federated Learning via Model Distillation

100

Characterizing Membership Privacy in Stochastic Gradient Langevin Dynamics

101

Alleviating Privacy Attacks via Causal Learning

102

MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples

103

Accident Risk Prediction based on Heterogeneous Sparse Data: New Dataset and Insights

104

GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models

105

On Inferring Training Data Attributes in Machine Learning Models

106

Federated Learning: Challenges, Methods, and Future Directions

107

Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection

108

Invariant Risk Minimization

109

On the Privacy Risks of Model Explanations

110

Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference

111

Generating Private Data Surrogates for Vision Related Tasks

112

Monte Carlo and Reconstruction Membership Inference Attacks against Generative Models

113

SocInf: Membership Inference Attacks on Social Media Health Data With Machine Learning

114

Disparate Vulnerability: on the Unfairness of Privacy Attacks Against Machine Learning

115

ML Defense: Against Prediction API Threats in Cloud-Based Machine Learning Service

116

White-box vs Black-box: Bayes Optimal Strategies for Membership Inference

117

Privacy Risks of Securing Machine Learning Models against Adversarial Examples

118

Membership Inference Attacks Against Adversarially Robust Deep Learning Models

119

The Audio Auditor: User-Level Membership Inference in Internet of Things Voice Services

120

Location Embeddings for Next Trip Recommendation

121

Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?

122

Evaluating Differentially Private Machine Learning in Practice

123

Measuring Membership Privacy on Aggregate Location Time-Series

124

GANobfuscator: Mitigating Information Leakage Under GAN via Differential Privacy

125

Demystifying Membership Inference Attacks in Machine Learning as a Service

126

Adversarial Attack and Defense on Graph Data: A Survey

127

Differentially Private Data Generative Models

128

Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning

129

Deep learning for healthcare: review, opportunities and challenges

130

Auditing Data Provenance in Text-Generation Models

131

Findings of the 2018 Conference on Machine Translation (WMT18)

132

Property Inference Attacks on Fully Connected Neural Networks using Permutation Invariant Representations

133

MLCapsule: Guarded Offline Deployment of Machine Learning as a Service

134

General Data Protection Regulation

135

Algorithms that remember: model inversion attacks and data protection law

136

Privacy-preserving Machine Learning through Data Obfuscation

137

Killing Four Birds with one Gaussian Process: The Relation between different Test-Time Attacks

138

ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

139

Performing Co-membership Attacks Against Deep Generative Models

140

BDD100K: A Diverse Driving Video Database with Scalable Annotation Tooling

141

Exploiting Unintended Feature Leakage in Collaborative Learning

142

Extreme Adaptation for Personalized Neural Machine Translation

143

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

144

Generating Artificial Data for Private Deep Learning

145

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks

146

Differentially Private Generative Adversarial Network

147

Understanding Membership Inferences on Well-Generalized Learning Models

148

Certified Robustness to Adversarial Examples with Differential Privacy

149

Machine Learning with Membership Privacy using Adversarial Regularization

150

Differentially Private Releasing via Deep Generative Model

151

Towards Measuring Membership Privacy

152

Differentially Private Federated Learning: A Client Level Perspective

153

Moonshine: Distilling with Cheap Convolutions

154

Progressive Growing of GANs for Improved Quality, Stability, and Variation

155

mixup: Beyond Empirical Risk Minimization

156

Learning Differentially Private Recurrent Language Models

157

The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes

158

Machine Learning Models that Remember Too Much

159

Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting

160

walk2friends: Inferring Social Links from Mobility Profiles

161

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

162

Knock Knock, Who's There? Membership Inference on Aggregate Location Data

163

WASSA-2017 Shared Task on Emotion Intensity

164

Privacy-Preserving Generative Deep Neural Networks Support Clinical Data Sharing

165

Towards Deep Learning Models Resistant to Adversarial Attacks

166

Attention is All you Need

167

LOGAN: Membership Inference Attacks Against Generative Models

168

ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases

169

Neural Collaborative Filtering

170

Improved Training of Wasserstein GANs

171

BEGAN: Boundary Equilibrium Generative Adversarial Networks

172

Generating Multi-label Discrete Patient Records using Generative Adversarial Networks

173

Age Progression/Regression by Conditional Adversarial Autoencoder

174

Rényi Differential Privacy

175

A survey on deep learning in medical image analysis

176

Towards the Science of Security and Privacy in Machine Learning

177

Understanding deep learning requires rethinking generalization

178

Adversarial Machine Learning at Scale

179

Membership Inference Attacks Against Machine Learning Models

180

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

181

Semi-Supervised Classification with Graph Convolutional Networks

182

Stealing Machine Learning Models via Prediction APIs

183

Enriching Word Vectors with Subword Information

184

node2vec: Scalable Feature Learning for Networks

185

Deep Learning with Differential Privacy

186

Multi-class texture analysis in colorectal cancer histology

187

Smart Reply: Automated Response Suggestion for Email

188

Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds

189

MIMIC-III, a freely accessible critical care database

190

The Cityscapes Dataset for Semantic Urban Scene Understanding

191

Communication-Efficient Learning of Deep Networks from Decentralized Data

192

Participatory Cultural Mapping Based on Collective Behavior Data in Location-Based Social Networks

193

Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases

194

Autoencoding beyond pixels using a learned similarity metric

195

Deep Residual Learning for Image Recognition

196

Rethinking the Inception Architecture for Computer Vision

197

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

198

Federated Optimization: Distributed Optimization Beyond the Datacenter

199

Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures

200

Train faster, generalize better: Stability of stochastic gradient descent

201

What Is Machine Learning

202

Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books

203

Distilling the Knowledge in a Neural Network

204

The human splicing code reveals new insights into the genetic determinants of disease

205

Explaining and Harnessing Adversarial Examples

206

Deep Learning Face Attributes in the Wild

207

GloVe: Global Vectors for Word Representation

208

A data-driven approach to cleaning large face datasets

209

Privacy in Pharmacogenetics: An End-to-End Case Study of Personalized Warfarin Dosing

210

CLiPS Stylometry Investigation (CSI) corpus: A Dutch corpus for the detection of age, gender, personality, sentiment and deception in text

211

Impact of HbA1c Measurement on Hospital Readmission Rates: Analysis of 70,000 Clinical Database Patient Records

212

Auto-Encoding Variational Bayes

213

Do Deep Nets Really Need to be Deep?

214

Intriguing properties of neural networks

215

Distributed Representations of Words and Phrases and their Compositionality

216

Hidden factors and hidden topics: understanding rating dimensions with review text

217

Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers

218

Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition

219

Bayesian Learning via Stochastic Gradient Langevin Dynamics

220

Chameleons in Imagined Conversations: A New Approach to Understanding Coordination of Linguistic Style in Dialogs

221

Caltech-UCSD Birds 200

222

On the Difficulties of Disclosure Prevention in Statistical Databases or The Case for Differential Privacy

223

ImageNet: A large-scale hierarchical image database

224

Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments

225

Collective Classification in Network Data

226

Resolving Individuals Contributing Trace Amounts of DNA to Highly Complex Mixtures Using High-Density SNP Genotyping Microarrays

227

Differential Privacy: A Survey of Results

228

Calibrating Noise to Sensitivity in Private Data Analysis

229

Acquiring linear subspaces for face recognition under variable lighting

230

RCV1: A New Benchmark Collection for Text Categorization Research

231

Unsupervised Learning: Foundations of Neural Computation

232

Monte Carlo Statistical Methods

233

NewsWeeder: Learning to Filter Netnews

234

Principles of Risk Minimization for Learning Theory

235

When Does Data Augmentation Help With Membership Inference Attacks?

236

Accuracy-Privacy Trade-off in Deep Ensembles

237

Comparing Local and Central Differential Privacy Using Membership Inference Attacks

238

Resisting membership inference attacks through knowledge distillation

239

Reconstruction-Based Membership Inference Attacks are Easier on Difficult Problems

240

GAN-Leaks: A Taxonomy ofMembership Inference Attacks against Generative Models. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (Virtual Event, USA) (CCS ’20)

241

Exploiting Transparency Measures for Membership Inference: a Cautionary Tale

242

Towards the Infeasibility of Membership

243

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

244

Membership Inference Attack against Differentially Private Deep Learning Model

245

ar X iv : 1 80 1 . 01 59 4 v 2 [ cs . C R ] 2 5 M ar 2 01 8 Differentially Private Releasing via Deep Generative Model ( Technical Report )

246

Philipp Koehn, and Christof Monz

247

Reddit comments dataset. https://bigquery.cloud.google.com/dataset/fh-bigquery:redditcomments

248

Weibo content corpus

249

Deep Learning

250

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

251

Dropout: a simple way to prevent neural networks from overfitting

252

Acquire Valued Shoppers Challenge

253

Information Theory and Statistics

254

Reading Digits in Natural Images with Unsupervised Feature Learning

255

Large text compression benchmark

256

Learning Multiple Layers of Features from Tiny Images

257

Texas Hospital Inpatient Discharge Public Use Data File

258

Texas Health Care Information Collection Center

259

The mnist database of handwritten digits

260

Artificial intelligence: a modern approach

261

Online algorithms and stochastic approximations

262

Gradient-based learning applied to document recognition

263

Long Short-TermMemory

264

Pattern recognition

265

Convergence de la répartition empirique vers la répartition théorique

266

Edinburgh Research Explorer Characteristic Functions on Graphs: Birds of a Feather, from Statistical Descriptors to Parametric Models

267

Neural network based attacks

268