Axiomatic Attribution for Deep Networks (2017-03-04T00:00:00.000000Z)

TL;DR

The problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works, is studied and two fundamental axioms— Sensitivity and Implementation Invariance that attribution methods ought to satisfy are identified.

Abstract

We study the problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works. We identify two fundamental axioms— Sensitivity and Implementation Invariance that attribution methods ought to satisfy. We show that they are not satisfied by most known attribution methods, which we consider to be a fundamental weakness of those methods. We use the axioms to guide the design of a new attribution method called Integrated Gradients. Our method requires no modification to the original network and is extremely simple to implement; it just needs a few calls to the standard gradient operator. We apply this method to a couple of image models, a couple of text models and a chemistry model, demonstrating its ability to debug networks, to extract rules from a network, and to enable users to engage with models better.

Authors

Ankur Taly

4 papers

Mukund Sundararajan

1 papers

Qiqi Yan

1 papers

TL;DR

Abstract

Authors

References35 items

Learning Important Features Through Propagating Activation Differences

Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs.

An unexpected unity among methods for interpreting model predictions

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Model-Agnostic Interpretability of Machine Learning

Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems

Not Just a Black Box: Learning Important Features Through Propagating Activation Differences

Layer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers

Learning executable semantic parsers for natural language understanding

Molecular graph convolutions: moving beyond fingerprints

“Why Should I Trust You?”: Explaining the Predictions of Any Classifier

Evaluating the Visualization of What a Deep Neural Network Has Learned

Compositional Semantic Parsing on Semi-Structured Tables

Understanding Neural Networks Through Deep Visualization

Inverting Visual Representations with Convolutional Networks

Striving for Simplicity: The All Convolutional Net

Explaining and Harnessing Adversarial Examples

Understanding deep image representations by inverting them

Going deeper with convolutions

Recurrent Neural Network Regularization

ImageNet Large Scale Visual Recognition Challenge

Neural Machine Translation by Jointly Learning to Align and Translate

Convolutional Neural Networks for Sentence Classification

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Visualizing and Understanding Convolutional Networks

Building high-level features using large scale unsupervised learning

How to Explain Individual Classification Decisions

Paths and consistency in additive cost sharing

Building a Large Annotated Corpus of English: The Penn Treebank

Values of Non-Atomic Games

The assignment game I: The core

Visualizing Higher-Layer Features of a Deep Network

Define function f : x ∈ [0, 1] n → R as 0 if min(x i , x j ) ≤ a, as (b − a) 2 if max(x i , x j ) ≥ b, and as (x i − a)(x j − a) otherwise. Next we compute the attributions of f at x = 1

LRP x 1 = 2 , x 2 = − 1

Axiomatic Attribution for Deep Networks

Field of Study

Venue Information

Name

Type

URL

Alternate Names