Deep Learning--based Text Classification (2020-04-06T00:00:00.000000Z)

TL;DR

A comprehensive review of more than 150 deep learning--based models for text classification developed in recent years is provided, and their technical contributions, similarities, and strengths are discussed.

Abstract

Deep learning--based models have surpassed classical machine learning--based approaches in various text classification tasks, including sentiment analysis, news categorization, question answering, and natural language inference. In this article, we provide a comprehensive review of more than 150 deep learning--based models for text classification developed in recent years, and we discuss their technical contributions, similarities, and strengths. We also provide a summary of more than 40 popular datasets widely used for text classification. Finally, we provide a quantitative analysis of the performance of different deep learning models on popular benchmarks, and we discuss future research directions.

Authors

Shervin Minaee

4 papers

E. Cambria

25 papers

Jianfeng Gao

21 papers

TL;DR

Abstract

Authors

References240 items

ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis

Robust Conversational AI with Grounded Text Generation

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

Language Models are Few-Shot Learners

MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification

Adversarial Training for Large Neural Language Models

XtremeDistil: Multi-stage Distillation for Massive Multilingual Models

Pre-trained models for natural language processing: A survey

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers

The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence

TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval

Image Segmentation Using Deep Learning: A Survey

Finding decision jumps in text classification

Biometric Recognition Using Deep Learning: A Survey

TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection

Adversarial NLI: A New Benchmark for Natural Language Understanding

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Text Level Graph Neural Network for Text Classification

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Improving text classification with weighted word embeddings via a multi-channel TextCNN model

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving

Semantics-aware BERT for Language Understanding

Neural Attentive Bag-of-Entities Model for Text Classification

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

ERNIE 2.0: A Continual Pre-training Framework for Language Understanding

Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment

Is BERT Really Robust? Natural Language Attack on Text Classification and Entailment

RoBERTa: A Robustly Optimized BERT Pretraining Approach

SpanBERT: Improving Pre-training by Representing and Predicting Spans

Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function

Investigating the transferring capability of capsule networks for text classification

Hierarchical Multi-label Classification of Text with Capsule Networks

XLNet: Generalized Autoregressive Pretraining for Language Understanding

Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification

Variational Pretraining for Semi-supervised Text Classification

Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications

How to Fine-Tune BERT for Text Classification?

Unified Language Model Pre-training for Natural Language Understanding and Generation

Unsupervised Data Augmentation for Consistency Training

Unsupervised Data Augmentation

ERNIE: Enhanced Representation through Knowledge Integration

Text Classification Algorithms: A Survey

Distilling Task-Specific Knowledge from BERT into Simple Neural Networks

Simplifying Graph Convolutional Networks

Multi-Task Deep Neural Networks for Natural Language Understanding

Squeezed Very Deep Convolutional Neural Networks for Text Classification

Contextualized Non-local Neural Networks for Sequence Learning

Compositional coding capsule network with k-means routing for text classification

Training Complex Models with Multi-Task Weak Supervision

Learning Universal Sentence Representations with Mean-Max Attention Autoencoder

Graph Convolutional Networks for Text Classification

Weakly-Supervised Neural Text Classification

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

Text Classification using Capsules

Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference

Densely Connected CNN with Multi-scale Feature Attention for Text Classification

Multiway Attention Networks for Modeling Sentence Pairs

Neural Approaches to Conversational AI

Enhancing Sentence Embedding with Generalized Pooling

Know What You Don’t Know: Unanswerable Questions for SQuAD

A Generative Model for category text generation

Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information

Joint Embedding of Words and Labels for Text Classification

Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

SciTaiL: A Textual Entailment Dataset from Science Question Answering

Targeted Aspect-Based Sentiment Analysis via Embedding Commonsense Knowledge into an Attentive LSTM

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN

Investigating Capsule Networks with Dynamic Routing for Text Classification

Matrix capsules with EM routing

Deep Contextualized Word Representations

Learning Structured Representation for Text Classification via Reinforcement Learning

Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling