Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations (2018-05-01T00:00:00.000000Z)

TL;DR

This work retrofit a language model with a label-conditional architecture, which allows the model to augment sentences without breaking the label-compatibility and improves classifiers based on the convolutional or recurrent neural networks.

Abstract

We propose a novel data augmentation for labeled sentences called contextual augmentation. We assume an invariance that sentences are natural even if the words in the sentences are replaced with other words with paradigmatic relations. We stochastically replace words with other words that are predicted by a bi-directional language model at the word positions. Words predicted according to a context are numerous but appropriate for the augmentation of the original words. Furthermore, we retrofit a language model with a label-conditional architecture, which allows the model to augment sentences without breaking the label-compatibility. Through the experiments for six various different text classification tasks, we demonstrate that the proposed method improves classifiers based on the convolutional or recurrent neural networks.

Authors

Sosuke Kobayashi

2 papers

TL;DR

Abstract

Authors

References43 items

Deep Contextualized Word Representations

MaskGAN: Better Text Generation via Filling in the ______

A Neural Language Model for Dynamically Representing the Meanings of Unknown Words and Entities in a Discourse

Data Augmentation for Visual Question Answering

Data Augmentation for Morphological Reinflection

Training Data Augmentation for Low-Resource Morphological Inflection

Dual Supervised Learning

Data Augmentation for Low-Resource Neural Machine Translation

Toward Controlled Generation of Text

Variational Autoencoder for Semi-Supervised Text Classification

Pointer Sentinel Mixture Models

context2vec: Learning Generic Context Embedding with Bidirectional LSTM

Sequence-Level Knowledge Distillation

Data Recombination for Neural Semantic Parsing

Dynamic Entity Representation with Max-pooling Improves Machine Reading

Improving Neural Machine Translation Models with Monolingual Data

Character-level Convolutional Networks for Text Classification

That’s So Annoying!!!: A Lexical and Frame-Semantic Embedding Based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors using #petpeeve Tweets

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

Transition-Based Dependency Parsing with Stack Long Short-Term Memory

Adam: A Method for Stochastic Optimization

Going deeper with convolutions

Sequence to Sequence Learning with Neural Networks

Convolutional Neural Networks for Sentence Classification

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

ImageNet classification with deep convolutional neural networks

Improving neural networks by preventing co-adaptation of feature detectors

Model-Portability Experiments for Textual Temporal Analysis

Semi-supervised Semantic Role Labeling Using the Latent Words Language Model

Semi-Supervised Semantic Role Labeling

Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts

Learning Question Classifiers

Long Short-Term Memory

WordNet: A Lexical Database for English

Cours de linguistique générale

GENERATIVE ADVERSARIAL NETS

Audio augmentation for speech recognition

Chainer : a Next-Generation Open Source Framework for Deep Learning

Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich

Vocal Tract Length Perturbation (VTLP) improves speech recognition

Annotating Expressions of Opinions and Emotions in Language

Transformation Invariance in Pattern Recognition-Tangent Distance and Tangent Propagation

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names