Jointly Learning to Label Sentences and Tokens (2018-11-14T00:00:00.000000Z)

TL;DR

This paper investigates how objectives at different granularities can be used to learn better language representations and proposes an architecture for jointly learning to label sentences and tokens and achieves substantial improvements for both sentence classification and sequence labeling.

Abstract

Learning to construct text representations in end-to-end systems can be difficult, as natural languages are highly compositional and task-specific annotated datasets are often limited in size. Methods for directly supervising language composition can allow us to guide the models based on existing knowledge, regularizing them towards more robust and interpretable representations. In this paper, we investigate how objectives at different granularities can be used to learn better language representations and we propose an architecture for jointly learning to label sentences and tokens. The predictions at each level are combined together using an attention mechanism, with token-level labels also acting as explicit supervision for composing sentence-level representations. Our experiments show that by learning to perform these tasks jointly on multiple levels, the model achieves substantial improvements for both sentence classification and sequence labeling.

Authors

Anders Søgaard

14 papers

Marek Rei

6 papers

TL;DR

Abstract

Authors

References42 items

Sequence Classification with Human Attention

Hierarchical Structured Model for Fine-to-Coarse Manifesto Text Analysis

Zero-Shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens

Grammatical Error Detection Using Error- and Grammaticality-Specific Word Embeddings

On the State of the Art of Evaluation in Neural Language Models

Convolutional Sequence to Sequence Learning

Semi-supervised Multitask Learning for Sequence Labeling

JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction

Language Modeling with Gated Convolutional Networks

Neural Machine Translation with Supervised Attention

Compositional Sequence Labeling Models for Error Detection in Learner Writing

Hierarchical Attention Networks for Document Classification

A Report on the Automatic Evaluation of Scientific Writing Shared Task

Rationale-Augmented Convolutional Neural Networks for Text Classification

Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss

Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection

Neural Architectures for Named Entity Recognition

Many Languages, One Parser

Neural Programmer: Inducing Latent Programs with Gradient Descent

Document Modeling with Gated Recurrent Neural Network for Sentiment Classification

Character-Aware Neural Language Models

Bidirectional LSTM-CRF Models for Sequence Tagging

A Hierarchical Neural Autoencoder for Paragraphs and Documents

GloVe: Global Vectors for Word Representation

Sequence to Sequence Learning with Neural Networks

Neural Machine Translation by Jointly Learning to Align and Translate

Convolutional Neural Networks for Sentence Classification

The CoNLL-2014 Shared Task on Grammatical Error Correction

Training and analyzing deep recurrent neural networks

Exploring the Utility of Joint Morphological and Syntactic Learning from Child-directed Speech

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Developing and testing a self-assessment and tutoring system

ADADELTA: An Adaptive Learning Rate Method

A New Dataset and Method for Automatically Grading ESOL Texts

The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text

Learning from Multiple Outlooks

Understanding the difficulty of training deep feedforward neural networks

The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes

Modeling Annotators: A Generative Approach to Learning from Annotator Rationales

Structured Models for Fine-to-Coarse Sentiment Analysis

Long Short-Term Memory

Dropout: a simple way to prevent neural networks from overfitting

Field of Study

Venue Information

Name

Type

URL

Alternate Names