End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF (2016-03-04T00:00:00.000000Z)

TL;DR

A novel neutral network architecture is introduced that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF, thus making it applicable to a wide range of sequence labeling tasks.

Abstract

State-of-the-art sequence labeling systems traditionally require large amounts of task-specific knowledge in the form of hand-crafted features and data pre-processing. In this paper, we introduce a novel neutral network architecture that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF. Our system is truly end-to-end, requiring no feature engineering or data pre-processing, thus making it applicable to a wide range of sequence labeling tasks. We evaluate our system on two data sets for two sequence labeling tasks --- Penn Treebank WSJ corpus for part-of-speech (POS) tagging and CoNLL 2003 corpus for named entity recognition (NER). We obtain state-of-the-art performance on both the two data --- 97.55\% accuracy for POS tagging and 91.21\% F1 for NER.

Authors

E. Hovy

15 papers

Xuezhe Ma

4 papers

TL;DR

Abstract

Authors

References64 items

Multi-Task Cross-Lingual Sequence Tagging from Scratch

Unsupervised Ranking Model for Entity Coreference Resolution

Neural Architectures for Named Entity Recognition

Harnessing Deep Neural Networks with Logic Rules

Improving Named Entity Recognition for Chinese Social Media with Word Segmentation Representation Learning

Named Entity Recognition with Bidirectional LSTM-CNNs

Joint Entity Recognition and Disambiguation

Non-lexical neural architecture for fine-grained POS Tagging

Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings

Efficient Inner-to-outer Greedy Algorithm for Higher-order Labeled Dependency Parsing

Bidirectional LSTM-CRF Models for Sequence Tagging

Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation

An Empirical Exploration of Recurrent Network Architectures

Transition-Based Dependency Parsing with Stack Long Short-Term Memory

Boosting Named Entity Recognition with Neural Character Embeddings

RMSProp and equilibrated adaptive learning rates for non-convex optimization

Probabilistic Models for High-Order Projective Dependency Parsing

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Enhancing of chemical compound and drug name recognition using representative tag scheme and fine-grained tokenization

Adam: A Method for Stochastic Optimization

Structure Regularization for Structured Prediction

GloVe: Global Vectors for Word Representation

On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

Learning Character-level Representations for Part-of-Speech Tagging

Unsupervised Dependency Parsing with Transferring Distribution via Parallel Guidance and Entropy Regularization

Lexicon Infused Phrase Embeddings for Named Entity Resolution

Distributed Representations of Words and Phrases and their Compositionality

Speech recognition with deep recurrent neural networks

ADADELTA: An Adaptive Learning Rate Method

Fourth-Order Dependency Parsing

On the difficulty of training recurrent neural networks

Semi-supervised condensed nearest neighbor for part-of-speech tagging

Part-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics?

Natural Language Processing (Almost) from Scratch

Efficient Third-Order Dependency Parsers

Supervised Noun Phrase Coreference Research: The First Fifteen Years

Understanding the difficulty of training deep feedforward neural networks

Phrase Clustering for Discriminative Learning

Design Challenges and Misconceptions in Named Entity Recognition

Guided Learning for Bidirectional Sequence Classification

A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data

Online Large-Margin Training of Dependency Parsers

2005 Special Issue: Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Deterministic Dependency Parsing of English Text

SVMTool: A general POS Tagger Generator Based on Support Vector Machines

Named Entity Recognition through Classifier Combination

Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition

Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network

Learning Precise Timing with LSTM Recurrent Networks

Named Entity Recognition: A Maximum Entropy Approach Using Global Information

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

Learning to Forget: Continual Prediction with LSTM

Representing Text Chunks

Long Short-Term Memory

Learning long-term dependencies with gradient descent is difficult

Building a Large Annotated Corpus of English: The Penn Treebank

Backpropagation Applied to Handwritten Zip Code Recognition

Dropout: a simple way to prevent neural networks from overfitting

A Fast and Accurate Dependency Parser using Neural Networks

CharWNN obtained near state-of-the-art accuracy on English POS tagging (see Section 4.3 for details)

Theano: a cpu and gpu math expression compiler

Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping

Learning task-dependent distributed representations by backpropagation through

gradient descent is difﬁcult

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names