MEIM: Multi-partition Embedding Interaction Beyond Block Term Format for Efficient and Expressive Link Prediction (2022-07-01T00:00:00.000000Z)

TL;DR

The Multi-partition Embedding Interaction iMproved beyond block term format (MEIM) model is introduced, with independent core tensor for ensemble effects and soft orthogonality for max-rank mapping, in addition to multi-partitions embedding.

Abstract

Knowledge graph embedding aims to predict the missing relations between entities in knowledge graphs. Tensor-decomposition-based models, such as ComplEx, provide a good trade-off between efficiency and expressiveness, that is crucial because of the large size of real world knowledge graphs. The recent multi-partition embedding interaction (MEI) model subsumes these models by using the block term tensor format and provides a systematic solution for the trade-off. However, MEI has several drawbacks, some of which carried from its subsumed tensor-decomposition-based models. In this paper, we address these drawbacks and introduce the Multi-partition Embedding Interaction iMproved beyond block term format (MEIM) model, with independent core tensor for ensemble effects and soft orthogonality for max-rank mapping, in addition to multi-partition embedding. MEIM improves expressiveness while still being highly efficient, helping it to outperform strong baselines and achieve state-of-the-art results on difficult link prediction benchmarks using fairly small embedding sizes. The source code is released at https://github.com/tranhungnghiep/MEIM.

Authors

A. Takasu

3 papers

Hung-Nghiep Tran

1 papers

TL;DR

Abstract

Authors

References43 items

Multi-Partition Embedding Interaction with Block Term Format for Knowledge Graph Completion

Generalized Translation-Based Embedding of Knowledge Graph

Low-Dimensional Hyperbolic Knowledge Graph Embeddings

You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings

A Re-evaluation of Knowledge Graph Completion Methods

Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding

Composition-based Multi-Relational Graph Convolutional Networks

InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions

Exploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space

Analyzing Knowledge Graph Embedding Methods from a Multi-Embedding Interaction Perspective

TuckER: Tensor Factorization for Knowledge Graph Completion

Information Geometry of Orthogonal Initializations and Training

RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

SimplE Embedding for Link Prediction in Knowledge Graphs

Convolutional 2D Knowledge Graph Embeddings

HyperNetworks

Complex Embeddings for Simple Link Prediction

Observed versus latent features for knowledge base and text inference

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

Translating Embeddings for Modeling Multi-relational Data

A Three-Way Model for Collective Learning on Multi-Relational Data

Tensor Decompositions and Applications

Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness

Boosting Algorithms as Gradient Descent

Some mathematical notes on three-mode factor analysis

RAGAT: Relation Aware Graph Attention Network for Knowledge Graph Completion

Multi-Relational Embedding for Knowledge Graph Representation and Analysis

SOKENDAI

Quaternion Knowledge Graph Embedding

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

In International Conference on Machine Learning

Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality

YAGO3: A Knowledge Base from Multilingual Wikipedias

Dropout: a simple way to prevent neural networks from overfitting

Theory and Practice of Digital Libraries

SIAM Review

Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. The Journal of Machine Learning Research

Zhiqing Sun, Shikhar Vashishth, Soumya Sanyal, Partha Talukdar, and Yiming Yang. A Reevaluation of Knowledge Graph Completion Methods

In Advances in Neural Information Processing Systems

Psychometrika

IEEE Transactions on Knowledge and Data Engineering

Field of Study

Venue Information

Name

Type

URL

Alternate Names