Span-based Joint Entity and Relation Extraction with Transformer Pre-training

Published in

European Conference on Artificial Intelligence(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

SpERT, an attention model for span-based joint entity and relation extraction, is introduced, which features entity recognition and filtering, as well as relation classification with a localized, marker-free context representation.

Abstract

We introduce SpERT, an attention model for span-based joint entity and relation extraction. Our key contribution is a light-weight reasoning on BERT embeddings, which features entity recognition and filtering, as well as relation classification with a localized, marker-free context representation. The model is trained using strong within-sentence negative samples, which are efficiently extracted in a single BERT pass. These aspects facilitate a search over all spans in the sentence. In ablation studies, we demonstrate the benefits of pre-training, strong negative sampling and localized context. Our model outperforms prior work by up to 2.6% F1 score on several datasets for joint entity and relation extraction.

Authors

Markus Eberts

1 papers

A. Ulges

1 papers

References47 items

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

Entity, Relation, and Event Extraction with Contextualized Span Representations

Enhancing Joint Entity and Relation Extraction with Language Modeling and Hierarchical Attention

RoBERTa: A Robustly Optimized BERT Pretraining Approach

SpanBERT: Improving Pre-training by Representing and Predicting Spans

Span-based Joint Entity and Relation Extraction with Transformer Pre-training

Published in

European Conference on Artificial Intelligence(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Markus Eberts

1 papers

A. Ulges

1 papers

References47 items

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

Entity, Relation, and Event Extraction with Contextualized Span Representations

Enhancing Joint Entity and Relation Extraction with Language Modeling and Hierarchical Attention

RoBERTa: A Robustly Optimized BERT Pretraining Approach

SpanBERT: Improving Pre-training by Representing and Predicting Spans

Span-Level Model for Relation Extraction

SemEval-2019 Task 3: EmoContext Contextual Emotion Detection in Text

Neural Metric Learning for Fast End-to-End Relation Extraction

Entity-Relation Extraction as Multi-Turn Question Answering

MASS: Masked Sequence to Sequence Pre-training for Language Generation

A general framework for information extraction using dynamic span graphs

SciBERT: A Pretrained Language Model for Scientific Text

Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers

End-to-End Open-Domain Question Answering with BERTserini

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

End-to-end neural relation extraction using deep biaffine attention

A Span Selection Model for Semantic Role Labeling

Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction

A Survey on Recent Advances in Named Entity Recognition from Deep Learning models

Adversarial training for multi-context joint entity and relation extraction

Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling

Joint entity recognition and relation extraction as a multi-head selection problem

Higher-Order Coreference Resolution with Coarse-to-Fine Inference

Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction

Deep Contextualized Word Representations

Joint Extraction of Multiple Relations and Entities by Using a Hybrid Neural Network

End-to-End Neural Relation Extraction with Global Optimization

End-to-end Neural Coreference Resolution

Attention is All you Need

Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme

A neural joint model for entity and relation extraction from biomedical text

Table Filling Multi-Task Recurrent Neural Network for Joint Entity and Relation Extraction

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Joint Models for Extracting Adverse Drug Events from Biomedical Text

End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures

Neural Machine Translation of Rare Words with Subword Units

Relation Classification via Recurrent Neural Network

Modeling Joint Entity and Relation Extraction with Table Representation

Neural Machine Translation by Jointly Learning to Align and Translate

Relation Classification via Convolutional Deep Neural Network

Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports

Semantic Compositionality through Recursive Matrix-Vector Spaces

A Linear Programming Formulation for Global Inference in Natural Language Tasks

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Improving Language Understanding by Generative Pre-Training

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

using 12 layers, 768-dimensional embeddings, 12 heads per layer, resulting in a total 110M parameters

Field of Study

Computer Science

Venue Information

Name

European Conference on Artificial Intelligence

Type

conference

URL

http://www.eccai.org/

Alternate Names

ECAI
Eur Conf Artif Intell
International Conference on Electronics, Computers and Artificial Intelligence
Int Conf Electron Comput Artif Intell
International Electronics, Computers and Artificial Intelligence
Int Electron Comput Artif Intell