Semi-supervised Multitask Learning for Sequence Labeling (2017-04-24T00:00:00.000000Z)

TL;DR

A sequence labeling framework with a secondary training objective, learning to predict surrounding words for every word in the dataset, which incentivises the system to learn general-purpose patterns of semantic and syntactic composition, useful for improving accuracy on different sequence labeling tasks.

Abstract

We propose a sequence labeling framework with a secondary training objective, learning to predict surrounding words for every word in the dataset. This language modeling objective incentivises the system to learn general-purpose patterns of semantic and syntactic composition, which are also useful for improving accuracy on different sequence labeling tasks. The architecture was evaluated on a range of datasets, covering the tasks of error detection in learner texts, named entity recognition, chunking and POS-tagging. The novel language modeling objective provided consistent performance improvements on every benchmark, without requiring any additional annotated or unannotated data.

Authors

Marek Rei

6 papers

TL;DR

Abstract

Authors

References36 items

Multi-Task Learning of Keyphrase Boundary Classification

Identifying beneficial task relations for multi-task learning in deep neural networks

Attending to Characters in Neural Sequence Labeling Models

Compositional Sequence Labeling Models for Error Detection in Learner Writing

Theano: A Python framework for fast computation of mathematical expressions

Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss

Neural Architectures for Named Entity Recognition

Open-Domain Name Error Detection using a Multi-Task RNN

Bidirectional LSTM-CRF Models for Sequence Tagging

CHEMDNER: The drugs and chemical names extraction challenge

Universal Dependencies 1.4

Opinion Mining with Deep Recurrent Neural Networks

The CoNLL-2014 Shared Task on Grammatical Error Correction

One billion word benchmark for measuring progress in statistical language modeling

Speech recognition with deep recurrent neural networks

Efficient Estimation of Word Representations in Vector Space

ADADELTA: An Adaptive Learning Rate Method

A fast and simple algorithm for training neural probabilistic language models

A New Dataset and Method for Automatically Grading ESOL Texts

Natural Language Processing (Almost) from Scratch

A unified architecture for natural language processing: deep neural networks with multitask learning

Developing a Robust Part-of-Speech Tagger for Biomedical Text

Exploring Deep Knowledge Resources in Biomedical Name Recognition

Introduction to the Bio-entity Recognition Task at JNLPBA

Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition

The GENIA corpus: an annotated research abstract corpus in molecular biology domain

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

Introduction to the CoNLL-2000 Shared Task Chunking

Long Short-Term Memory

Multitask Learning

Building a Large Annotated Corpus of English: The Penn Treebank

Dropout: a simple way to prevent neural networks from overfitting

Ilya Sutskever, and Ruslan Salakhutdinov

Recurrent neural network based language model

Hierarchical Probabilistic Neural Network Language Model

Tjong Kim Sang and Fien De Meulder

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names