Robust Design of Deep Neural Networks Against Adversarial Attacks Based on Lyapunov Theory (2019-11-12T00:00:00.000000Z)

TL;DR

This work treats each individual layer of the DNN as a nonlinear system and uses Lyapunov theory to prove stability and robustness locally and globally, and develops empirically tight bounds on the response of the output layer, or any hidden layer, to adversarial perturbations added to the input, or the input of hidden layers.

Abstract

Deep neural networks (DNNs) are vulnerable to subtle adversarial perturbations applied to the input. These adversarial perturbations, though imperceptible, can easily mislead the DNN. In this work, we take a control theoretic approach to the problem of robustness in DNNs. We treat each individual layer of the DNN as a nonlinear system and use Lyapunov theory to prove stability and robustness locally. We then proceed to prove stability and robustness globally for the entire DNN. We develop empirically tight bounds on the response of the output layer, or any hidden layer, to adversarial perturbations added to the input, or the input of hidden layers. Recent works have proposed spectral norm regularization as a solution for improving robustness against l2 adversarial attacks. Our results give new insights into how spectral norm regularization can mitigate the adversarial effects. Finally, we evaluate the power of our approach on a variety of data sets and network architectures and against some of the well-known adversarial attacks.

Authors

Edward Raff

7 papers

Arash Rahnama

1 papers

A. Nguyen

1 papers

TL;DR

Abstract

Authors

References40 items

Connecting Lyapunov Control Theory to Adversarial Attacks

The coupling effect of Lipschitz regularization in deep neural networks

Certified Adversarial Robustness via Randomized Smoothing

Intermediate Level Adversarial Attack for Enhanced Transferability

Efficient Neural Network Robustness Certification with General Activation Functions

Stability-Certified Reinforcement Learning: A Control-Theoretic Perspective

Generalizable Adversarial Training via Spectral Normalization

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

Unrestricted Adversarial Examples

Robustness May Be at Odds with Accuracy

The Singular Values of Convolutional Layers

Towards Fast Computation of Certified Robustness for ReLU Networks

Regularisation of neural networks by enforcing Lipschitz continuity

Adversarial vulnerability for any classifier

L2-Nonexpansive Neural Networks

Adversarial Spheres

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Robust Linear Regression Against Training Data Poisoning

Provable defenses against adversarial examples via the convex outer adversarial polytope

One Pixel Attack for Fooling Deep Neural Networks

Efficient Defenses Against Adversarial Attacks

Towards Deep Learning Models Resistant to Adversarial Attacks

Spectral Norm Regularization for Improving the Generalizability of Deep Learning

Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods

Parseval Networks: Improving Robustness to Adversarial Examples

Maximum Resilience of Artificial Neural Networks

Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks

Technical Report on the CleverHans v2.1.0 Adversarial Examples Library

Towards Evaluating the Robustness of Neural Networks

Adversarial examples in the physical world

Deep Residual Learning for Image Recognition

Distributional Smoothing with Virtual Adversarial Training

Explaining and Harnessing Adversarial Examples

Intriguing properties of neural networks

Multiple classifier systems for robust classifier design in adversarial environments

Matrix diagonal stability in systems and computation

On the input-output stability of time-varying nonlinear feedback systems Part one: Conditions derived using concepts of loop gain, conicity, and positivity

A Recurring Theorem on Determinants

Nonlinear Systems

This Paper Is Included in the Proceedings of the 12th Usenix Symposium on Operating Systems Design and Implementation (osdi '16). Tensorflow: a System for Large-scale Machine Learning Tensorflow: a System for Large-scale Machine Learning

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names