Membership Inference Attacks From First Principles (2021-12-07T00:00:00.000000Z)

TL;DR

A Likelihood Ratio Attack is developed that is $10\times more powerful at low false-positive rates, and also strictly dominates prior attacks on existing metrics.

Abstract

A membership inference attack allows an adversary to query a trained machine learning model to predict whether or not a particular example was contained in the model’s training dataset. These attacks are currently evaluated using average-case “accuracy” metrics that fail to characterize whether the attack can confidently identify any members of the training set. We argue that attacks should instead be evaluated by computing their true-positive rate at low (e.g., ≤ 0.1%) false-positive rates, and find most prior attacks perform poorly when evaluated in this way. To address this we develop a Likelihood Ratio Attack (LiRA) that carefully combines multiple ideas from the literature. Our attack is $10\times$ more powerful at low false-positive rates, and also strictly dominates prior attacks on existing metrics.

Authors

Florian Tramèr

5 papers

Milad Nasr

5 papers

Nicholas Carlini

2 papers

TL;DR

Abstract

Authors

References78 items

Enhanced Membership Inference Attacks against Machine Learning Models

On the Importance of Difficulty Calibration in Membership Inference Attacks

Understanding deep learning (still) requires rethinking generalization

ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models

Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning

Extracting Training Data from Large Language Models

When is memorization of irrelevant training data necessary for high-accuracy learning?

A Pragmatic Approach to Membership Inferences on Machine Learning Models

Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries

What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation

Membership Leakage in Label-Only Exposures

Label-Only Membership Inference Attacks

ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning

Auditing Differentially Private Machine Learning: How Private is Private SGD?

Revisiting Membership Inference Under Realistic Assumptions

Stealing Links from Graph Neural Networks

Systematic Evaluation of Privacy Risks of Machine Learning Models

MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples

GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models

Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference

Does learning require memorization? a short tale about a long tail

White-box vs Black-box: Bayes Optimal Strategies for Membership Inference

Privacy Risks of Securing Machine Learning Models against Adversarial Examples

Gmail Smart Compose: Real-Time Assisted Writing

Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning

Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning

Property Inference Attacks on Fully Connected Neural Networks using Permutation Invariant Representations

Towards Demystifying Membership Inference Attacks

ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

AutoAugment: Learning Augmentation Policies from Data

Exploiting Unintended Feature Leakage in Collaborative Learning

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks

Scalable Private Learning with PATE

Stealing Hyperparameters in Machine Learning

Machine Learning with Membership Privacy using Adversarial Regularization

Towards Measuring Membership Privacy

Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting

Knock Knock, Who's There? Membership Inference on Aggregate Location Data

Random Erasing Data Augmentation

LOGAN: Membership Inference Attacks Against Generative Models

Exposed! A Survey of Attacks on Private Data

Dermatologist-level classification of skin cancer with deep neural networks

Membership Inference Attacks Against Machine Learning Models

Pointer Sentinel Mixture Models

SGDR: Stochastic Gradient Descent with Warm Restarts

Deep Learning with Differential Privacy

Wide Residual Networks

Deep Residual Learning for Image Recognition

Robust Traceability from Trace Amounts

Better Malware Ground Truth: Techniques for Weighting Anti-Virus Vendor Labels

Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures

The Algorithmic Foundations of Differential Privacy

Stochastic gradient descent with differentially private updates

The Composition Theorem for Differential Privacy

ImageNet: A large-scale hierarchical image database

A Statistical Framework for Differential Privacy

Resolving Individuals Contributing Trace Amounts of DNA to Highly Complex Mixtures Using High-Density SNP Genotyping Microarrays

Learning to Detect and Classify Malicious Executables in the Wild

The Art of Data Augmentation

A Simple Weight Decay Can Improve Generalization

Increased rates of convergence through learning rate adaptation

Works

Quantifying the Privacy Risks of Learning High-Dimensional Graphical Models

huyvnphan/pytorch cifar10, January 2021

Introducing a new privacy testing library in tensorﬂow

Poster: Ultimate Power of Inference Attacks: Privacy Risks of Learning High-Dimensional Graphical Models

Language Models are Unsupervised Multitask Learners

Membership Inference Attack against Differentially Private Deep Learning Model

Detecting Credential Spearphishing in Enterprise Settings

Learning Multiple Layers of Features from Tiny Images

Genomic privacy and limits of individual detection in a pool

Spam Filtering with Naive Bayes - Which Naive Bayes?

A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection

Gradient-based learning applied to document recognition

SpamCop: A Spam Classification & Organisation Program

On the Problem of the Most Efficient Tests of Statistical Hypotheses