How Interpretable and Trustworthy are GAMs? (2020-06-11T00:00:00.000000Z)

TL;DR

It is found that GAMs with high feature sparsity can miss patterns in the data and be unfair to rare subpopulations, and tree-based GAMs represent the best balance of sparsity, fidelity and accuracy and thus appear to be the most trustworthy GAM models.

Abstract

Generalized additive models (GAMs) have become a leading model class for interpretable machine learning. However, there are many algorithms for training GAMs, and these can learn different or even contradictory models, while being equally accurate. Which GAM should we trust? In this paper, we quantitatively and qualitatively investigate a variety of GAM algorithms on real and simulated datasets. We find that GAMs with high feature sparsity (only using a few variables to make predictions) can miss patterns in the data and be unfair to rare subpopulations. Our results suggest that inductive bias plays a crucial role in what interpretable models learn and that tree-based GAMs represent the best balance of sparsity, fidelity and accuracy and thus appear to be the most trustworthy GAM models.

Authors

R. Caruana

8 papers

A. Goldenberg

5 papers

Benjamin J. Lengerich

2 papers

TL;DR

Abstract

Authors

References49 items

The optimal cut-off point of vitamin D for pregnancy outcomes using a generalized additive model.

Generalized Additive Models to Capture the Death Rates in Canada COVID-19

Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods

InterpretML: A Unified Framework for Machine Learning Interpretability

A Survey on Bias and Fairness in Machine Learning

Generalized additive models: Building evidence of air pollution, climate change and human health.

Hierarchical generalized additive models in ecology: an introduction with mgcv

Please Stop Permuting Features: An Explanation and Alternatives

Unbiased Measurement of Feature Importance in Tree-Based Methods

Splitting on categorical predictors in random forests

Learning Global Additive Explanations for Neural Nets Using Model Distillation

pyGAM: Generalized Additive Models in Python

Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation

Applying generalized additive models to unravel dynamic changes in anthocyanin biosynthesis in methyl jasmonate elicited grapevine (Vitis vinifera cv. Gamay) cell cultures

A Unified Approach to Interpreting Model Predictions

Towards A Rigorous Science of Interpretable Machine Learning

Additive models with trend filtering

Have We Substantially Underestimated the Impact of Improved Sanitation Coverage on Child Health? A Generalized Additive Model Panel Analysis of Global Data on Child Mortality and Malnutrition

Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments

False Positives, False Negatives, and False Analyses: A Rejoinder to "Machine Bias: There's Software Used across the Country to Predict Future Criminals. and It's Biased against Blacks"

The mythos of model interpretability

MIMIC-III, a freely accessible critical care database

XGBoost: A Scalable Tree Boosting System

“Why Should I Trust You?”: Explaining the Predictions of Any Classifier

Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission

Fused Lasso Additive Model

Learning Fair Representations

Generalized additive models in business and economics

Intelligible models for classification and regression

Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models

On Feature Selection, Bias-Variance, and Bagging

A comparison of methods for the fitting of generalized additive models

Decision Tree Instability and Active Learning

The elements of statistical learning: data mining, inference and prediction

Sparsity and smoothness via the fused lasso

AMlet, RAMlet, and GAMlet: Automatic Nonlinear Fitting of Additive Models, Robust and Generalized, With Wavelets

An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid

Generalized additive models for medical research

Spline Models for Observational Data

Generalized Additive Models

A Comparison of GCV and GML for Choosing the Smoothing Parameter in the Generalized Spline Smoothing Problem

An Evaluation of the Doctor-Interpretability of Generalized Additive Models with Interactions

mboost: Model-Based Boosting

UCI Machine Learning Repository

BMC Bioinformatics BioMed Central Methodology article

Regression Shrinkage and Selection via the Lasso

© Institute of Mathematical Statistics, 2010 To Explain or to Predict?

A REPRODUCIBILITY: TRAINING DETAILS, HYPERPARAMETERS, AND DATASETS

Field of Study

Journal Information

Name

Venue Information

Name

Type

URL

Alternate Names