Hierarchically-Refined Label Attention Network for Sequence Labeling (2019-08-23T00:00:00.000000Z)

TL;DR

A hierarchically-refined label attention network is investigated, which explicitly leverages label embeddings and captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention.

Abstract

CRF has been used as a powerful model for statistical sequence labeling. For neural sequence labeling, however, BiLSTM-CRF does not always lead to better results compared with BiLSTM-softmax local classification. This can be because the simple Markov label transition model of CRF does not give much information gain over strong neural encoding. For better representing label sequences, we investigate a hierarchically-refined label attention network, which explicitly leverages label embeddings and captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention. Results on POS tagging, NER and CCG supertagging show that the proposed model not only improves the overall tagging accuracy with similar number of parameters, but also significantly speeds up the training and testing compared to BiLSTM-CRF.

Authors

Leyang Cui

3 papers

Yue Zhang

7 papers

TL;DR

Abstract

Authors

References67 items

Benchmarking Approximate Inference Methods for Neural Structured Prediction

Star-Transformer

Learning Better Internal Structure of Words for Sequence Labeling

Semi-Supervised Sequence Modeling with Cross-View Training

One vs. Many QA Matching with both Word-level and Sentence-level Attention Network

Two Local Models for Neural Constituent Parsing

Learning Tag Dependencies for Sequence Tagging

Design Challenges and Misconceptions in Neural Sequence Labeling

NCRF++: An Open-source Neural Sequence Labeling Toolkit

Robust Lexical Features for Improved Neural Network Named-Entity Recognition

Cross-Target Stance Classification with Self-Attention Networks

Joint Embedding of Words and Labels for Text Classification

Sentence-State LSTM for Text Representation

Linguistically-Informed Self-Attention for Semantic Role Labeling

Learning Approximate Inference Networks for Structured Prediction

Robust Multilingual Part-of-Speech Tagging via Adversarial Training

Multi-Task Label Embedding for Text Classification

Empower Sequence Labeling with Task-Aware Neural Language Model

Recurrent Attention Network on Memory for Aspect Sentiment Analysis

Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks

Deep Active Learning for Named Entity Recognition

Attention is All you Need

Attention Modeling for Targeted Sentiment

Universal Dependencies 2.1

Fast and Accurate Entity Recognition with Iterated Dilated Convolutions

Deep Biaffine Attention for Neural Dependency Parsing

Discourse Parsing with Attention-based Hierarchical Neural Networks

Deep multi-task learning with low level tasks supervised at lower layers

Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification

Supertagging With LSTMs

LSTM CCG Parsing

Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss

Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

Text Understanding with the Attention Sum Reader Network

Neural Architectures for Named Entity Recognition

All-in Text: Learning Document, Label, and Word Representations Jointly

Named Entity Recognition with Bidirectional LSTM-CNNs

Zero-Shot Learning via Joint Latent Similarity Embedding

A Neural Attention Model for Abstractive Sentence Summarization

Bidirectional LSTM-CRF Models for Sequence Tagging

PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks

CCG Supertagging with a Recurrent Neural Network

End-to-end learning of semantic role labeling using recurrent neural networks

Teaching Machines to Read and Comprehend

A Joint Model for Entity Analysis: Coreference, Typing, and Linking

GloVe: Global Vectors for Word Representation

Neural Machine Translation by Jointly Learning to Align and Translate

Learning Character-level Representations for Part-of-Speech Tagging

Towards Robust Linguistic Analysis using OntoNotes

Polyglot: Distributed Word Representations for Multilingual NLP

Zero-Shot Learning Through Cross-Modal Transfer

CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes

Part-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics?

Zero-shot Learning with Semantic Output Codes

Design Challenges and Misconceptions in Named Entity Recognition

CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank

OntoNotes: The 90% Solution

Chinese Segmentation and New Word Detection using Conditional Random Fields

Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network

‘It's Skin Cancer’… a Rollercoaster of a Journey for Teenagers, Young People and Their Significant Other

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

Building a Large Annotated Corpus of English: The Penn Treebank

Phrase-level Self-Attention Networks for Universal Sentence Encoding

Empty Category Detection With Joint Context-Label Embeddings

A Gold Standard Dependency Corpus for English

Conference of the North American Chapter of the Association

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names