TOWARDS HIERARCHICAL IMPORTANCE ATTRIBUTION: EXPLAINING COMPOSITIONAL SEMANTICS FOR NEURAL SEQUENCE MODELS (2020-01-01T00:00:00.000000Z)

TL;DR

This paper proposes a formal and general way to quantify the importance of each word and phrase and proposes Sampling and Contextual Decomposition (SCD) algorithm and Sampling and Occlusion (SOC) algorithm, which outperform prior hierarchical explanation algorithms.

Abstract

The impressive performance of neural networks on natural language processing tasks attributes to their ability to model complicated word and phrase compositions. To explain how the model handles semantic compositions, we study hierarchical explanation of neural network predictions. We identify non-additivity and context independent importance attributions within hierarchies as two desirable properties for highlighting word and phrase compositions. We show some prior efforts on hierarchical explanations, e.g. contextual decomposition, do not satisfy the desired properties mathematically, leading to inconsistent explanation quality in different models. In this paper, we start by proposing a formal and general way to quantify the importance of each word and phrase. Following the formulation, we propose Sampling and Contextual Decomposition (SCD) algorithm and Sampling and Occlusion (SOC) algorithm. Human and metrics evaluation on both LSTM models and BERT Transformer models on multiple datasets show that our algorithms outperform prior hierarchical explanation algorithms. Our algorithms help to visualize semantic composition captured by models, extract classification rules and improve human trust of models. Project page: this https URL

Authors

X. Xue

5 papers

Xiang Ren

7 papers

Xisen Jin

2 papers

TL;DR

Abstract

Authors

References51 items

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

RoBERTa: A Robustly Optimized BERT Pretraining Approach

BERT Rediscovers the Classical NLP Pipeline

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems

A Survey of Evaluation Methods and Measures for Interpretable Machine Learning

Towards Explainable NLP: A Generative Explanation Framework for Text Classification

Can I trust you more? Model-Agnostic Hierarchical Explanations

L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data

Explaining Image Classifiers by Counterfactual Generation

Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement

Hierarchical interpretations for neural network predictions

Learning to Explain: An Information-Theoretic Perspective on Model Interpretation

Deep Contextualized Word Representations

A Survey of Methods for Explaining Black Box Models

Evaluating neural network explanation methods using hybrid documents and morphological agreement

Evaluating neural network explanation methods using hybrid documents and morphological prediction

Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs

Towards better understanding of gradient-based attribution methods for Deep Neural Networks

A unified view of gradient-based attribution methods for Deep Neural Networks

Position-aware Attention and Supervised Data Improve Slot Filling

Attention is All you Need

A Unified Approach to Interpreting Model Predictions

Detecting Statistical Interactions from Neural Network Weights

Learning Important Features Through Propagating Activation Differences

Axiomatic Attribution for Deep Networks

Visualizing Deep Neural Network Decisions: Prediction Difference Analysis

Visualizing the effects of predictor variables in black box supervised learning models

Understanding Neural Networks through Representation Erasure

Interpretation of Prediction Models Using the Input Gradient

Rationalizing Neural Predictions

Layer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers

Representation of Linguistic Form and Function in Recurrent Neural Networks

“Why Should I Trust You?”: Explaining the Predictions of Any Classifier

Character-level Convolutional Networks for Text Classification

GloVe: Global Vectors for Word Representation

Neural Machine Translation by Jointly Learning to Align and Translate

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

The Feature Importance Ranking Measure

Detecting statistical interactions with additive groves of trees

Axiomatic characterizations of probabilistic and cardinal-probabilistic interaction indices

Discovering additive structure in black box functions

Long Short-Term Memory

A Value for n-person Games

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

our algorithms interpret context dependent interactions and can be applied as a building block for calculating interaction index such as Shapley Interactions

F1 score on average. We use the same parameter settings between LSTM classiﬁers and language models on three datasets

Interpreting Context Dependent Interactions

ACD computes the β term for the output linear layer h (cid:48) = W h + b as

interpreting how two phrases interacts in a spe-ciﬁc context

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names