Quantifying Explainability of Saliency Methods in Deep Neural Networks With a Synthetic Dataset (2020-09-07T00:00:00.000000Z)

TL;DR

A synthetic dataset that can be generated adhoc along with the ground-truth heatmaps for more objective quantitative assessment of different XAI methods is introduced and mabCAM is introduced as the heatmap generation method compatible with the authors' ground- Truth heatmaps.

Abstract

Post-hoc analysis is a popular category in eXplainable artificial intelligence (XAI) study. In particular, methods that generate heatmaps have been used to explain the deep neural network (DNN), a black-box model. Heatmaps can be appealing due to the intuitive and visual ways to understand them but assessing their qualities might not be straightforward. Different ways to assess heatmaps' quality have their own merits and shortcomings. This article introduces a synthetic dataset that can be generated adhoc along with the ground-truth heatmaps for more objective quantitative assessment. Each sample data is an image of a cell with easily recognized features that are distinguished from localization ground-truth mask, hence facilitating a more transparent assessment of different XAI methods. Comparison and recommendations are made, shortcomings are clarified along with suggestions for future research directions to handle the finer details of select posthoc analysis methods. Furthermore, mabCAM is introduced as the heatmap generation method compatible with our ground-truth heatmaps. The framework is easily generalizable and uses only standard deep learning components.

Authors

Cuntai Guan

4 papers

Erico Tjoa

1 papers

TL;DR

Abstract

Authors

References64 items

XAI Meets Mobile Traffic Classification: Understanding and Improving Multimodal Deep Learning Architectures

An Experimental Study of Quantitative Evaluations on Saliency Methods

Explainable Internet Traffic Classification

SWAG: Superpixels Weighted by Average Gradients for Explanations of CNNs

Quantitative Evaluations on Saliency Methods: An Experimental Study

Understanding the role of individual units in a deep neural network

There and Back Again: Revisiting Backpropagation Saliency Methods

CLEVR-XAI: A benchmark dataset for the ground truth evaluation of neural network explanations

Explainable AI for medical imaging: deep-learning CNN ensemble for classification of estrogen receptor status from breast MRI

When Explanations Lie: Why Many Modified BP Attributions Fail

Enhancing the Extraction of Interpretable Information for Ischemic Stroke Imaging from Deep Neural Networks

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI

Guideline-Based Additive Explanation for Computer-Aided Diagnosis of Lung Nodules

Testing the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer's disease classification

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI

Generation of Multimodal Justification Using Visual Word Constraint Model for Explainable Computer-Aided Diagnosis

CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipeline

Analyzing Neuroimaging Data Through Recurrent Deep Learning Models

Sanity Checks for Saliency Maps

Brain Biomarker Interpretation in ASD Using Deep Learning and fMRI

A Benchmark for Interpretability Methods in Deep Neural Networks

Multiple Instance Learning for Heterogeneous Images: Training a CNN for Histopathology

Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations

Explaining Explanations: An Overview of Interpretability of Machine Learning

Autofocus Layer for Semantic Segmentation

Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks

Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)

Towards better understanding of gradient-based attribution methods for Deep Neural Networks

Interpreting Deep Visual Representations via Network Dissection

SmoothGrad: removing noise by adding noise

Attention is All you Need

A Unified Approach to Interpreting Model Predictions

Network Dissection: Quantifying Interpretability of Deep Visual Representations

Interpretable Explanations of Black Boxes by Meaningful Perturbation

Learning Important Features Through Propagating Activation Differences

Axiomatic Attribution for Deep Networks

Not Just a Black Box: Learning Important Features Through Propagating Activation Differences

“Why Should I Trust You?”: Explaining the Predictions of Any Classifier

Learning Deep Features for Discriminative Localization

Deep Residual Learning for Image Recognition

Very deep convolutional neural network based image classification using small training sample size

Evaluating the Visualization of What a Deep Neural Network Has Learned

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

Striving for Simplicity: The All Convolutional Net

One weird trick for parallelizing convolutional neural networks

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Visualizing and Understanding Convolutional Networks

Volume under the ROC Surface for Multi-class Problems

Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support: Second International Workshop, iMIMIC 2019, and 9th International Workshop, ML-CDS 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Proceedings

In-terpretability of Machine Intelligence

Generalizability vs. robustness: In-vestigating medical imaging networks using adversarial examples

Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization

Èöóóööñññòò Êê×××ö Öóùô

LRP tutorial

A research into the role of negative values in the heatmaps may be necessary

Noise term. Noise added to d can be found in pipeline

The heatmaps produced by ResNet appear to be sparsest, followed by VGG then AlexNet

Cottrell , Antonio Criminisi , and Aditya V . Nori . Autofocus layer for semantic segmentation . CoRR , abs / 1805 . 08403 , 2018

– MICCAI 2018

Tutorial LRP

Thresholds th d , th l = 0 . 05 are suitably chosen to create binary arrays

license agreement with IEEE. Restrictions apply

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names