Towards Evaluating the Robustness of Neural Networks (2016-08-16T00:00:00.000000Z)

TL;DR

It is demonstrated that defensive distillation does not significantly increase the robustness of neural networks, and three new attack algorithms are introduced that are successful on both distilled and undistilled neural networks with 100% probability are introduced.

Abstract

Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input x and any target classification t, it is possible to find a new input x' that is similar to x but classified as t. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from 95% to 0.5%.In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with 100% probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples.

Authors

Nicholas Carlini

1 papers

D. Wagner

1 papers

TL;DR

Abstract

Authors

References56 items

Thermometer Encoding: One Hot Way To Resist Adversarial Examples

A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots

Safety Verification of Deep Neural Networks

Cleverhans V0.1: an Adversarial Machine Learning Library

Fast, Lean, and Accurate: Modeling Password Guessability Using Neural Networks

Hidden Voice Commands

On the Effectiveness of Defensive Distillation

Adversarial examples in the physical world

Adversarial Perturbations Against Deep Neural Networks for Malware Classification

Measuring Neural Net Robustness with Constraints

Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples

End to End Learning for Self-Driving Cars

Globally Normalized Transition-Based Neural Networks

Accelerating Wright–Fisher Forward Simulations on the Graphics Processing Unit

Mastering the game of Go with deep neural networks and tree search

Deep Residual Learning for Image Recognition

Rethinking the Inception Architecture for Computer Vision

The Limitations of Deep Learning in Adversarial Settings

Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

All you need is a good init

Understanding adversarial training: Increasing local stability of supervised models through robust optimization

Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks

Learning with a Strong Adversary

Malware classification with recurrent networks

Distilling the Knowledge in a Neural Network

Human-level control through deep reinforcement learning

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Striving for Simplicity: The All Convolutional Net

Explaining and Harnessing Adversarial Examples

Fractional Max-Pooling

Towards Deep Neural Network Architectures Robust to Adversarial Examples

ImageNet Large Scale Visual Recognition Challenge

Droid-Sec

Intriguing properties of neural networks

Playing Atari with Deep Reinforcement Learning

Large-scale malware classification using random projections and neural networks

Speech recognition with deep recurrent neural networks

ImageNet classification with deep convolutional neural networks

Deep Neural Networks for Acoustic Modeling in Speech Recognition

Anomaly detection: A survey

ImageNet: A large-scale hierarchical image database

Neural Networks in Mobile Robot Motion

Safety Veriﬁcation of Deep Neural Networks ∗

Bringing big neural networks to self-driving cars, smartphones, and drones

Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization

Rectifier Nonlinearities Improve Neural Network Acoustic Models

FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597

Learning Multiple Layers of Features from Tiny Images

The mnist database of handwritten digits

GradientBased Learning Applied to Document Recognition

Gradient-based learning applied to document recognition

throw out the ﬁrst network and use only the soft training labels

Compute soft labels by apply the teacher network to each instance in the training set, again evaluating the softmax at temperature T

Train the distilled network (a network with the same shape as the teacher network) on the soft labels, using softmax at temperature T

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names