Efficient Neural Network Robustness Certification with General Activation Functions (2018-11-01T00:00:00.000000Z)

TL;DR

This paper introduces CROWN, a general framework to certify robustness of neural networks with general activation functions for given input data points and facilitates the search for a tighter certified lower bound by adaptively selecting appropriate surrogates for each neuron activation.

Abstract

Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem. Nevertheless, recently it has been shown to be possible to give a non-trivial certified lower bound of minimum adversarial distortion, and some recent progress has been made towards this direction by exploiting the piece-wise linear nature of ReLU activations. However, a generic robustness certification for general activation functions still remains largely unexplored. To address this issue, in this paper we introduce CROWN, a general framework to certify robustness of neural networks with general activation functions for given input data points. The novelty in our algorithm consists of bounding a given activation function with linear and quadratic functions, hence allowing it to tackle general activation functions including but not limited to four popular choices: ReLU, tanh, sigmoid and arctan. In addition, we facilitate the search for a tighter certified lower bound by adaptively selecting appropriate surrogates for each neuron activation. Experimental results show that CROWN on ReLU networks can notably improve the certified lower bounds compared to the current state-of-the-art algorithm Fast-Lin, while having comparable computational efficiency. Furthermore, CROWN also demonstrates its effectiveness and flexibility on networks with general activation functions, including tanh, sigmoid and arctan.

Authors

Cho-Jui Hsieh

11 papers

Pin-Yu Chen

17 papers

Huan Zhang

3 papers

TL;DR

Abstract

Authors

References33 items

AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation

Towards Fast Computation of Certified Robustness for ReLU Networks

A Dual Approach to Scalable Verification of Deep Networks

Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach

Certified Defenses against Adversarial Examples

Deep Neural Networks as 0-1 Mixed Integer Linear Programs: A Feasibility Study

Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models

Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning

Show-and-Fool: Crafting Adversarial Examples for Neural Image Captioning

The Robustness of Deep Networks: A Geometrical Perspective

Provable defenses against adversarial examples via the convex outer adversarial polytope

Provably Minimally-Distorted Adversarial Examples

Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification

EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples

ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models

Robust Physical-World Attacks on Machine Learning Models

Synthesizing Robust Adversarial Examples

An approach to reachability analysis for feed-forward ReLU neural networks

Towards Deep Learning Models Resistant to Adversarial Attacks

Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation

Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks

Maximum Resilience of Artificial Neural Networks

Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks

Delving into Transferable Adversarial Examples and Black-box Attacks

Towards Evaluating the Robustness of Neural Networks

Adversarial examples in the physical world

Practical Black-Box Attacks against Machine Learning

DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks

Explaining and Harnessing Adversarial Examples

Intriguing properties of neural networks

Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification

Lower bounds on the robustness to adversarial perturbations

Certifiable Distributional Robustness with Principled Adversarial Training

Field of Study

Venue Information

Name

Type

URL

Alternate Names