Certified Defenses for Data Poisoning Attacks (2017-06-09T00:00:00.000000Z)

TL;DR

This work addresses the worst-case loss of a defense in the face of a determined attacker by constructing approximate upper bounds on the loss across a broad family of attacks, for defenders that first perform outlier removal followed by empirical risk minimization.

Abstract

Machine learning systems trained on user-provided data are susceptible to data poisoning attacks, whereby malicious users inject false training data with the aim of corrupting the learned model. While recent work has proposed a number of attacks and defenses, little is understood about the worst-case loss of a defense in the face of a determined attacker. We address this by constructing approximate upper bounds on the loss across a broad family of attacks, for defenders that first perform outlier removal followed by empirical risk minimization. Our approximation relies on two assumptions: (1) that the dataset is large enough for statistical concentration between train and test error to hold, and (2) that outliers within the clean (non-poisoned) data do not have a strong effect on the model. Our bound comes paired with a candidate attack that often nearly matches the upper bound, giving us a powerful tool for quickly assessing defenses on a given dataset. Empirically, we find that even under a simple defense, the MNIST-1-7 and Dogfish datasets are resilient to attack, while in contrast the IMDB sentiment dataset can be driven from 12% to 23% test error by adding only 3% poisoned data.

Authors

Percy Liang

37 papers

Pang Wei Koh

9 papers

J. Steinhardt

10 papers

TL;DR

Abstract

Authors

References67 items

SoK: Security and Privacy in Machine Learning

Resilient Linear Classification: An Approach to Deal with Attacks on Training Data

Analysis of Causative Attacks against SVMs Learning from Data Streams

Understanding Black-box Predictions via Influence Functions

Tactics of Adversarial Attack on Deep Reinforcement Learning Agents

Generative Poisoning Attack Method Against Neural Networks

Adversarial Attacks on Neural Network Policies

Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks

On the Security of Machine Learning in Malware C&C Detection

Towards the Science of Security and Privacy in Machine Learning

Learning from untrusted data

Data Poisoning Attacks on Factorization-Based Collaborative Filtering

Stealing Machine Learning Models via Prediction APIs

Hidden Voice Commands

Adversarial examples in the physical world

Avoiding Imposters and Delinquents: Adversarial Crowdsourcing and Peer Prediction

Curie: A method for protecting SVM Classifier from Poisoning Attack

Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples

Agnostic Estimation of Mean and Covariance

Robust Estimators in High Dimensions without the Computational Intractability

Modeling Adversarial Learning as Nested Stackelberg Games

CVXPY: A Python-Embedded Modeling Language for Convex Optimization

The Teaching Dimension of Linear Learners

Systematic Poisoning Attacks on and Defenses for Machine Learning in Healthcare

Support vector machines under adversarial label contamination

Is Feature Selection Secure against Training Data Poisoning?

Robust Regression via Hard Thresholding

The Security of Latent Dirichlet Allocation

Using Machine Teaching to Identify Optimal Training-Set Attacks on Machine Learners

Explaining and Harnessing Adversarial Examples

The Statistics of Streaming Sparse Regression

On the Practicality of Integrity Attacks on Document-Level Sentiment Analysis

Poisoning behavioral malware clustering

Poisoning Complete-Linkage Hierarchical Clustering

Practical Evasion of a Learning-Based Classifier: A Case Study

Security Evaluation of Pattern Classifiers under Attack

Intriguing properties of neural networks

Is data clustering in adversarial settings secure?

The power of localization for efficiently learning linear separators with noise

Robust High Dimensional Sparse Regression and Matching Pursuit

Adversarial Label Flips Attack on Support Vector Machines

Poisoning Attacks against Support Vector Machines

Online Learning and Online Convex Optimization

Robust Lasso With Missing and Grossly Corrupted Observations

Stackelberg games for adversarial prediction problems

Learning Word Vectors for Sentiment Analysis

Exact Recoverability From Dense Corrupted Observations via $\ell _{1}$-Minimization

The security of machine learning

Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization

ANTIDOTE: understanding and defending against poisoning of anomaly detectors

Learning Halfspaces with Malicious Noise

On the Complexity of Linear Prediction: Risk Bounds, Margin Bounds, and Regularization

Casting out Demons: Sanitizing Training Data for Anomaly Sensors

Paragraph: Thwarting Signature Learning by Training Maliciously

Diagnosing network-wide traffic anomalies

The Art and Science of Computer Security

Practical Bilevel Optimization: Algorithms and Applications

GENERATIVE ADVERSARIAL NETS

Gurobi Optimization, Inc

Combating Attacks and Abuse in Large Online Communities

Static prediction games for adversarial learning problems

How Much Spam Can You Take? An Analysis of Crowdsourcing Results to Increase Accuracy

Let `(θ;x, y) be any margin-based loss: `(θ;x, y) = φ(y〈θ, x〉), where φ is 1-Lipschitz. Then the bound (14) holds with probability 1 − δ, for E(N

Spam Filtering with Naive Bayes - Which Naive Bayes?

YALMIP : a toolbox for modeling and optimization in MATLAB

Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones

La cryptographie militaire

Field of Study

Venue Information

Name

Type

URL

Alternate Names