VOICE: Variance of Induced Contrastive Explanations to Quantify Uncertainty in Neural Network Interpretability

Published in

IEEE Journal on Selected Topics in Signal Proce...(2024)

External Links:

Generate Graph DownloadPDF

TL;DR

This analysis allows for a plug in approach to visualize and quantify the remaining predictive uncertainty of any gradient-based explanatory technique, and shows that every image, network, prediction, and explanatory technique has a unique uncertainty.

Abstract

In this paper, we visualize and quantify the predictive uncertainty of gradient-based post hoc visual explanations for neural networks. Predictive uncertainty refers to the variability in the network predictions under perturbations to the input. Visual post hoc explainability techniques highlight features within an image to justify a network's prediction. We theoretically show that existing evaluation strategies of visual explanatory techniques partially reduce the predictive uncertainty of neural networks. This analysis allows us to construct a plug in approach to visualize and quantify the remaining predictive uncertainty of any gradient-based explanatory technique. We show that every image, network, prediction, and explanatory technique has a unique uncertainty. The proposed uncertainty visualization and quantification yields two key observations. Firstly, oftentimes under incorrect predictions, explanatory techniques are uncertain about the same features that they are attributing the predictions to, thereby reducing the trustworthiness of the explanation. Secondly, objective metrics of an explanation's uncertainty, empirically behave similarly to epistemic uncertainty. We support these observations on two datasets, four explanatory techniques, and six neural network architectures.

Authors

M. Prabhushankar

7 papers

Ghassan AlRegib

1 papers

References66 items

Clinically Labeled Contrastive Learning for OCT Biomarker Classification

Probing the Purview of Neural Networks via Gradient Analysis

Gaussian Switch Sampling: A Second-Order Approach to Active Learning

Stochastic surprisal: An inferential measurement of free energy in neural networks

On the Ramifications of Human Label Uncertainty

VOICE: Variance of Induced Contrastive Explanations to Quantify Uncertainty in Neural Network Interpretability

Published in

IEEE Journal on Selected Topics in Signal Proce...(2024)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

M. Prabhushankar

7 papers

Ghassan AlRegib

1 papers

References66 items

Clinically Labeled Contrastive Learning for OCT Biomarker Classification

Probing the Purview of Neural Networks via Gradient Analysis

Gaussian Switch Sampling: A Second-Order Approach to Active Learning

Stochastic surprisal: An inferential measurement of free energy in neural networks

On the Ramifications of Human Label Uncertainty

Gradient-Based Severity Labeling for Biomarker Classification in Oct

Introspective Learning : A Two-Stage Approach for Inference in Neural Networks

Reliable uncertainty estimation for seismic interpretation with prediction switches

Gradient-Based Adversarial and Out-of-Distribution Detection

Explanatory Paradigms in Neural Networks: Towards relevant and contextual explanations

A survey of uncertainty in deep neural networks

Contrastive Reasoning in Neural Networks

Extracting Causal Visual Features For Limited Label Classification

Gradients as a Measure of Uncertainty in Neural Networks

Novelty Detection Through Model-Based Characterization of Neural Networks

Implicit Saliency In Deep Neural Networks

Contrastive Explanations In Neural Networks

Backpropagated Gradient Representations for Anomaly Detection

Fabric surface characterization: assessment of deep learning-based texture representations using a challenging dataset

How Should We Quantify Uncertainty in Statistical Inference?

Distorted Representation Space Characterization Through Backpropagated Gradients

U-CAM: Visual Explanation Using Uncertainty Based Class Activation Maps

Uncertainty Based Detection and Relabeling of Noisy Image Labels

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

Towards understanding common features between natural and seismic images

RISE: Randomized Input Sampling for Explanation of Black-box Models

A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations

Understanding Measures of Uncertainty for Adversarial Example Detection

Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)

Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks

Interpreting CNN knowledge via an Explanatory Graph

On Calibration of Modern Neural Networks

SmoothGrad: removing noise by adding noise

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Densely Connected Convolutional Networks

UNIQUE: Unsupervised Image Quality Estimation

Discovering Causal Signals in Images

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size

“Why Should I Trust You?”: Explaining the Predictions of Any Classifier

Learning Deep Features for Discriminative Localization

Deep Residual Learning for Image Recognition

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Striving for Simplicity: The All Convolutional Net

Explaining and Harnessing Adversarial Examples

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet classification with deep convolutional neural networks

ImageNet: A large-scale hierarchical image database

Disagreement as a Measure of Uncertainty

Evaluating Prediction Uncertainty

Learning representations by back-propagating errors

with layered intermediate predictions

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

“Reliable post hoc explanations:Modelinguncertaintyinexplainability,”in

“Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning,”

Introduction to the Cooperative Special Issue on Graph Signal Processing in the IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS and the IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING

Uncertainty in Deep Learning

Explaining Explainers: Necessity and Sufficiency in Tabular Data

and trust, and human in the loop algorithms

Excellence award, all in

group has demonstrated their work on a wide range of applications such as autonomous systems, medical imaging, and subsurface imaging

systems in real-world scenarios

“Transitional uncertainty with intermediate neural gaussian processes.”

ma-chinelearning,activelearning,healthcare,androbustandexplainableAI

McCarty Chair Professor in the School of Electrical and Computer Engineering at the Georgia Institute of Technology

Field of Study

Computer Science

Journal Information

Name

IEEE Journal of Selected Topics in Signal Processing

Page

168-179

Volume

Venue Information

Name

IEEE Journal on Selected Topics in Signal Processing

Type

journal

URL

http://ieeexplore.ieee.org/servlet/opac?punumber=4200690

Alternate Names

IEEE J Sel Top Signal Process
IEEE Journal of Selected Topics in Signal Processing