ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection (2021-08-27T00:00:00.000000Z)

TL;DR

Experimental results show that the proposed ProtoInfoMax method can substantially improve performance up to 20% for OOD detection in low resource settings of text classification, and is less prone to typical overconfidence errors of Neural Networks, leading to more reliable prediction results.

Abstract

The ability to detect Out-of-Domain (OOD) inputs has been a critical requirement in many real-world NLP applications. For example, intent classification in dialogue systems. The reason is that the inclusion of unsupported OOD inputs may lead to catastrophic failure of systems. However, it remains an empirical question whether current methods can tackle such problems reliably in a realistic scenario where zero OOD training data is available. In this study, we propose ProtoInfoMax, a new architecture that extends Prototypical Networks to simultaneously process in-domain and OOD sentences via Mutual Information Maximization (InfoMax) objective. Experimental results show that our proposed method can substantially improve performance up to 20% for OOD detection in low resource settings of text classification. We also show that ProtoInfoMax is less prone to typical overconfidence errors of Neural Networks, leading to more reliable prediction results.

Authors

Mykola Pechenizkiy

7 papers

Iftitahu Ni'mah

2 papers

Meng Fang

2 papers

TL;DR

Abstract

Authors

References51 items

A statistical interpretation of term specificity and its application in retrieval

A Neural Few-Shot Text Classification Reality Check

A Two-phase Prototypical Network Model for Incremental Few-shot Relation Classification

Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference

Intent Mining from past conversations for Conversational Agent

Efficient Intent Detection with Dual Sentence Encoders

Distance-Based Learning from Errors for Confidence Calibration

Hierarchical Attention Prototypical Networks for Few-Shot Text Classification

A Mutual Information Maximization Perspective of Language Representation Learning

FewRel 2.0: Towards More Challenging Few-Shot Relation Classification

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction

QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization

Few-shot Text Classification with Distributional Signatures

Out-of-Domain Detection for Low-Resource Text Classification Tasks

On Mutual Information Maximization for Representation Learning

OCGAN: One-Class Novelty Detection Using GANs With Constrained Latent Representations

Learning Speaker Representations with Mutual Information

Recent Advances in Open Set Recognition: A Survey

Adversarially Learned Anomaly Detection

FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation

Generative Question Answering: Learning to Answer the Whole Question

Deep Anomaly Detection with Outlier Exposure

A Less Biased Evaluation of Out-of-distribution Sample Detectors

Learning deep representations by mutual information estimation and maximization

Deep One-Class Classification

Joint Learning of Domain Classification and Out-of-Domain Detection with Dynamic Class Weighting for Satisficing False Acceptance Rates

Diverse Few-Shot Text Classification with Multiple Metrics

Mutual Information Neural Estimation

DOC: Deep Open Classification of Text Documents

Learning how to Active Learn: A Deep Reinforcement Learning Approach

On Calibration of Modern Neural Networks

Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks

Prototypical Networks for Few-shot Learning

Neural sentence embedding using only in-domain sentences for out-of-domain sentence detection in dialog systems

Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

Learning Cumulatively to Become More Knowledgeable

Breaking the Closed World Assumption in Text Classification

A Diversity-Promoting Objective Function for Neural Conversation Models

Obtaining Well Calibrated Probabilities Using Bayesian Binning

Detecting out-of-domain utterances addressed to a virtual personal assistant

One-class classification: taxonomy of study and review of techniques

Toward Open Set Recognition

Out-of-Domain Utterance Detection Using Classification Confidences of Multiple Topics

Predicting good probabilities with supervised learning

Estimating the Support of a High-Dimensional Distribution

An Information-Maximization Approach to Blind Separation and Blind Deconvolution

Term-Weighting Approaches in Automatic Text Retrieval

Self-organization in a perceptual network

Out-of-domain Detection based on Generative Adversarial Network

Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Field of Study

Venue Information

Name

Type

URL

Alternate Names