A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection

Published in

arXiv.org(2021)

External Links:

Generate Graph

TL;DR

On a wide selection of challenging vision, language, and biology OOD benchmarks, it is shown that RMD meaningfully improves upon MD performance (by up to 15% AUROC on genomics OOD).

Abstract

Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks. We analyze its failure modes for near-OOD detection and propose a simple fix called relative Mahalanobis distance (RMD) which improves performance and is more robust to hyperparameter choice. On a wide selection of challenging vision, language, and biology OOD benchmarks (CIFAR-100 vs CIFAR-10, CLINC OOD intent detection, Genomics OOD), we show that RMD meaningfully improves upon MD performance (by up to 15% AUROC on genomics OOD).

Authors

Abhijit Guha Roy

5 papers

Balaji Lakshminarayanan

8 papers

Jie Jessie Ren

4 papers

References25 items

Exploring the Limits of Out-of-Distribution Detection

Vision Transformers are Robust Learners

Learning Transferable Visual Models From Natural Language Supervision

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Contrastive Training for Improved Out-of-Distribution Detection

A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection

Published in

arXiv.org(2021)

External Links:

Generate Graph

TL;DR

On a wide selection of challenging vision, language, and biology OOD benchmarks, it is shown that RMD meaningfully improves upon MD performance (by up to 15% AUROC on genomics OOD).

Abstract

Authors

Abhijit Guha Roy

5 papers

Balaji Lakshminarayanan

8 papers

Jie Jessie Ren

4 papers

References25 items

Exploring the Limits of Out-of-Distribution Detection

Vision Transformers are Robust Learners

Learning Transferable Visual Models From Natural Language Supervision

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Contrastive Training for Improved Out-of-Distribution Detection

Stanislav Fort

3 papers

Shreyas Padhy

3 papers

Density of States Estimation for Out-of-Distribution Detection

Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness

Hybrid Models for Open Set Recognition

Why is the Mahalanobis Distance Effective for Anomaly Detection?

Big Transfer (BiT): General Visual Representation Learning

Normalizing Flows for Probabilistic Modeling and Inference

Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality

An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction

Likelihood Ratios for Out-of-Distribution Detection

Using Pre-Training Can Improve Model Robustness and Uncertainty

Deep Anomaly Detection with Outlier Exposure

A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks

Masked Autoregressive Flow for Density Estimation

A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

Concrete Problems in AI Safety

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Novelty detection and neural network validation

Fit the background Gaussian N ( µ 0 , Σ 0 ) using D intrain ignoring the class labels, where µ 0 = 1 N (cid:80) Ni =1 z i and Σ 0 = 1 N (cid:80) Ni =1 ( z i − µ 0 ) ( z i − µ 0 ) T

5 We compare our method with the Partial Mahalanobis distance (PMD) proposed in [9]. PMD uses a subset of eigen-bases to compute the distance score

Compute MD 0 ( z (cid:48) ) based on N ( µ 0 , Σ 0 ) , for each z (cid:48) ∈

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

arXiv.org

Type

URL

https://arxiv.org

Alternate Names

ArXiv