KG-BERT: BERT for Knowledge Graph Completion
Published in
arXiv.org(2019)External Links:
TL
TL;DR
This work treats triples in knowledge graphs as textual sequences and proposes a novel framework named Knowledge Graph Bidirectional Encoder Representations from Transformer (KG-BERT) to model these triples.
Abstract
Knowledge graphs are important resources for many artificial intelligence tasks but often suffer from incompleteness. In this work, we propose to use pre-trained language models for knowledge graph completion. We treat triples in knowledge graphs as textual sequences and propose a novel framework named Knowledge Graph Bidirectional Encoder Representations from Transformer (KG-BERT) to model these triples. Our method takes entity and relation descriptions of a triple as input and computes scoring function of the triple with the KG-BERT language model. Experimental results on multiple benchmark knowledge graphs show that our method can achieve state-of-the-art performance in triple classification, link prediction and relation prediction tasks.
References44 items
1
A convolutional neural network-based model for knowledge base completion and its application to search personalization
2
XLNet: Generalized Autoregressive Pretraining for Language Understanding
3
A Multiscale Visualization of Attention in the Transformer Model
4
COMET: Commonsense Transformers for Automatic Knowledge Graph Construction
5