Interpretability and Explainability: A Machine Learning Zoo Mini-tour (2020-12-03T00:00:00.000000Z)

TL;DR

This review examines the problem of designing interpretable and explainable machine learning models and emphasises the divide between interpretability and explainability and illustrates these two different research directions with concrete examples of the state-of-the-art.

Abstract

In this review, we examine the problem of designing interpretable and explainable machine learning models. Interpretability and explainability lie at the core of many machine learning and statistical applications in medicine, economics, law, and natural sciences. Although interpretability and explainability have escaped a clear universal definition, many techniques motivated by these properties have been developed over the recent 30 years with the focus currently shifting towards deep learning methods. In this review, we emphasise the divide between interpretability and explainability and illustrate these two different research directions with concrete examples of the state-of-the-art. The review is intended for a general machine learning audience with interest in exploring the problems of interpretation and explanation beyond logistic regression or random forest variable importance. This work is not an exhaustive literature survey, but rather a primer focusing selectively on certain lines of research which the authors found interesting or informative.

Authors

Ricards Marcinkevics

2 papers

Julia E. Vogt

3 papers

TL;DR

Abstract

Authors

References123 items

A survey of algorithmic recourse: definitions, formulations, solutions, and prospects

Concept Bottleneck Models

Enhanced Integrated Gradients: improving interpretability of deep learning models using splicing codes as a case study

Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data

Algorithmic recourse under imperfect causal knowledge: a probabilistic approach

DeepCoDA: personalized interpretability for compositional health data

Revealing the Phase Diagram of Kitaev Materials by Machine Learning: Cooperation and Competition between Spin Liquids

A Survey of Deep Learning for Scientific Discovery

Explainable k-Means and k-Medians Clustering

Problems with Shapley-value-based explanations as feature importance measures

Algorithmic Recourse: from Counterfactual Explanations to Interventions

Discovering Nonlinear Relations with Minimum Predictive Information Regularization

Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers

Economy Statistical Recurrent Units For Inferring Nonlinear Granger Causality

Bayesian Symbolic Regression

Tracing diagnosis trajectories over millions of patients reveal an unexpected risk in schizophrenia

DPSOM: Deep Probabilistic Clustering with Self-Organizing Maps

Generalized Integrated Gradients: A practical method for explaining diverse ensembles

A study in Rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning

Machine Learning Interpretability: A Survey on Methods and Metrics

Generative Counterfactual Introspection for Explainable Deep Learning

Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions

Meaning to Form: Measuring Systematicity as Information

Disentangled State Space Representations

Is Attention Interpretable?

AI Feynman: A physics-inspired method for symbolic regression

Model-Agnostic Counterfactual Explanations for Consequential Decisions

Explaining machine learning classifiers through diverse counterfactual explanations

Faculty Opinions recommendation of Panning for gold: ‘model‐X’ knockoffs for high dimensional controlled variable selection.

Full-Gradient Representation for Neural Network Visualization

Attention is not Explanation

Causal Discovery with Attention-Based Convolutional Neural Networks

An introduction to domain adaptation and transfer learning

Recent Advances in Autoencoder-Based Representation Learning

Learning Influence Structure in Sparse Social Networks

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

Deep Knockoffs

KnockoffGAN: Generating Knockoffs for Feature Selection using Generative Adversarial Networks

DeepPINK: reproducible feature selection in deep neural networks

Explaining Image Classifiers by Counterfactual Generation

Discovering Interpretable Representations for Both Deep Generative and Discriminative Models

Towards Robust Interpretability with Self-Explaining Neural Networks

How Important Is a Neuron?

Disentangled Sequential Autoencoder

Neural Granger Causality for Nonlinear Time Series

Granger-Causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks

Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classification

The Doctor Just Won't Accept That!

Statistical genetic programming for symbolic regression

Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR

An optimization approach to learning falling rule lists

Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data

Optimized Risk Scores

Attention is All you Need

Contextual Explanation Networks

A Unified Approach to Interpreting Model Predictions

Learning Important Features Through Propagating Activation Differences

Axiomatic Attribution for Deep Networks

Towards A Rigorous Science of Interpretable Machine Learning

beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Towards deep learning with segregated dendrites

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

The mythos of model interpretability

“Why Should I Trust You?”: Explaining the Predictions of Any Classifier

Causal Falling Rule Lists

Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

Supersparse linear integer models for optimized medical scoring systems

Generative Moment Matching Networks

Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks

The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification

Falling Rule Lists

Inductive Logic Programming

Controlling the false discovery rate via knockoffs