TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents (2019-03-01T00:00:00.000000Z)

TL;DR

This work focuses on Trojan attacks that augment the function of reinforcement learning policies with hidden behaviors that can be implemented through minuscule data poisoning and in-band reward modification that does not affect the reward on normal inputs.

Abstract

Recent work has identified that classification models implemented as neural networks are vulnerable to data-poisoning and Trojan attacks at training time. In this work, we show that these training-time vulnerabilities extend to deep reinforcement learning (DRL) agents and can be exploited by an adversary with access to the training process. In particular, we focus on Trojan attacks that augment the function of reinforcement learning policies with hidden behaviors. We demonstrate that such attacks can be implemented through minuscule data poisoning (as little as 0.025% of the training data) and in-band reward modification that does not affect the reward on normal inputs. The policies learned with our proposed attack approach perform imperceptibly similar to benign policies but deteriorate drastically when the Trojan is triggered in both targeted and untargeted settings. Furthermore, we show that existing Trojan defense mechanisms for classification tasks are not effective in the reinforcement learning setting.

Authors

Susmit Jha

3 papers

Panagiota Kiourti

1 papers

Kacper Wardega

1 papers

TL;DR

Abstract

Authors

References34 items

Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering

Spectral Signatures in Backdoor Attacks

Physical Adversarial Examples for Object Detectors

Adversarial Perturbations Against Real-Time Video Classification Systems

Robust Physical-World Attacks on Deep Learning Visual Classification

Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Audio Adversarial Examples: Targeted Attacks on Speech-to-Text

Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

Adversarial Examples that Fool Detectors

Toward Human Parity in Conversational Speech Recognition

Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents

BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

Robust Physical-World Attacks on Deep Learning Models

Efficient Parallel Methods for Deep Reinforcement Learning

Tactics of Adversarial Attack on Deep Reinforcement Learning Agents

Adversarial Attacks on Neural Network Policies

Delving into Transferable Adversarial Examples and Black-box Attacks

Towards Evaluating the Robustness of Neural Networks

Adversarial examples in the physical world

End to End Learning for Self-Driving Cars

Practical Black-Box Attacks against Machine Learning

Mastering the game of Go with deep neural networks and tree search

The Limitations of Deep Learning in Adversarial Settings

High-Dimensional Continuous Control Using Generalized Advantage Estimation

Human-level control through deep reinforcement learning

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Explaining and Harnessing Adversarial Examples

Deep neural networks are easily fooled: High confidence predictions for unrecognizable images

Intriguing properties of neural networks

Trojaning Attack on Neural Networks

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names