Stronger data poisoning attacks break data sanitization defenses (2018-11-02T00:00:00.000000Z)

TL;DR

Three attacks are developed that can bypass a broad range of common data sanitization defenses, including anomaly detectors based on nearest neighbors, training loss, and singular-value decomposition, and the Karush–Kuhn–Tucker conditions.

Abstract

Machine learning models trained on data from the outside world can be corrupted by data poisoning attacks that inject malicious points into the models’ training sets. A common defense against these attacks is data sanitization: first filter out anomalous training points before training the model. In this paper, we develop three attacks that can bypass a broad range of common data sanitization defenses, including anomaly detectors based on nearest neighbors, training loss, and singular-value decomposition. By adding just 3% poisoned data, our attacks successfully increase test error on the Enron spam detection dataset from 3 to 24% and on the IMDB sentiment classification dataset from 12 to 29%. In contrast, existing attacks which do not explicitly account for these data sanitization defenses are defeated by them. Our attacks are based on two ideas: (i) we coordinate our attacks to place poisoned points near one another, and (ii) we formulate each attack as a constrained optimization problem, with constraints designed to ensure that the poisoned points evade detection. As this optimization involves solving an expensive bilevel problem, our three attacks correspond to different ways of approximating this problem, based on influence functions; minimax duality; and the Karush–Kuhn–Tucker (KKT) conditions. Our results underscore the need to develop more robust defenses against data poisoning attacks.

Authors

Percy Liang

37 papers

Pang Wei Koh

9 papers

J. Steinhardt

10 papers

TL;DR

Abstract

Authors

References97 items

Generative Adversarial Networks

Adversarial machine learning

Proposition

Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks

Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks

SoK: Security and Privacy in Machine Learning

Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks

Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning

Sever: A Robust Meta-Algorithm for Stochastic Optimization

Attack Strength vs. Detectability Dilemma in Adversarial Machine Learning

Detection of Adversarial Training Examples in Poisoning Attacks through Anomaly Detection

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Certified Defenses against Adversarial Examples

Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

Robust Linear Regression Against Training Data Poisoning

Provable defenses against adversarial examples via the convex outer adversarial polytope

Certifying Some Distributional Robustness with Principled Adversarial Training

Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization

BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

Learning geometric concepts with nasty noise

Towards Deep Learning Models Resistant to Adversarial Attacks

Certified Defenses for Data Poisoning Attacks

Ensemble Adversarial Training: Attacks and Defenses

Analysis of Causative Attacks against SVMs Learning from Data Streams

Resilience: A Criterion for Learning in the Presence of Arbitrary Outliers

Understanding Black-box Predictions via Influence Functions

Generative Poisoning Attack Method Against Neural Networks

Being Robust (in High Dimensions) Can Be Practical

On the Security of Machine Learning in Malware C&C Detection

Towards the Science of Security and Privacy in Machine Learning

Learning from untrusted data

Data Poisoning Attacks on Factorization-Based Collaborative Filtering

Hidden Voice Commands

Adversarial examples in the physical world

Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples

Agnostic Estimation of Mean and Covariance

Robust Estimators in High Dimensions without the Computational Intractability

Second Order Stochastic Optimization in Linear Time

Practical Black-Box Attacks against Machine Learning

Tight Bounds for Approximate Carathéodory and Beyond

DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks

Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

Support vector machines under adversarial label contamination

The Security of Latent Dirichlet Allocation

Using Machine Teaching to Identify Optimal Training-Set Attacks on Machine Learners

Explaining and Harnessing Adversarial Examples

On the Practicality of Integrity Attacks on Document-Level Sentiment Analysis

Security Evaluation of Pattern Classifiers under Attack

Intriguing properties of neural networks

The power of localization for efficiently learning linear separators with noise

Poisoning attacks to compromise face templates

Poisoning Adaptive Biometric Systems

Adversarial Label Flips Attack on Support Vector Machines

Poisoning Attacks against Support Vector Machines

Notes About the Carathéodory Number

Support Vector Machines Under Adversarial Label Noise

Learning Word Vectors for Sentiment Analysis

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

The security of machine learning

Deep learning via Hessian-free optimization

Security analysis of online centroid anomaly detection

ANTIDOTE: understanding and defending against poisoning of anomaly detectors

Learning Halfspaces with Malicious Noise

On Agnostic Learning of Parities, Monomials, and Halfspaces

Support vector machines

Casting out Demons: Sanitizing Training Data for Anomaly Sensors

Exploiting Machine Learning to Subvert Your Spam Filter

Hardness of Learning Halfspaces with Noise

On the Learnability and Design of Output Codes for Multiclass Problems

Practical Bilevel Optimization: Algorithms and Applications

Fast Exact Multiplication by the Hessian

Learning in the presence of malicious errors

Some properties of the bilevel programming problem

Forms

Robust learning: information theory and algorithms