AutoSF: Searching Scoring Functions for Knowledge Graph Embedding (2019-04-26T00:00:00.000000Z)

TL;DR

Results on link prediction and triplets classification show that the searched SFs by AutoSF, are KG dependent, new to the literature, and outperform the state-of- the-art SFs designed by humans.

Abstract

Scoring functions (SFs), which measure the plausibility of triplets in knowledge graph (KG), have become the crux of KG embedding. Lots of SFs, which target at capturing different kinds of relations in KGs, have been designed by humans in recent years. However, as relations can exhibit complex patterns that are hard to infer before training, none of them can consistently perform better than others on existing benchmark data sets. In this paper, inspired by the recent success of automated machine learning (AutoML), we propose to automatically design SFs (AutoSF) for distinct KGs by the AutoML techniques. However, it is non-trivial to explore domain- specific information here to make AutoSF efficient and effective. We firstly identify a unified representation over popularly used SFs, which helps to set up a search space for AutoSF. Then, we propose a greedy algorithm to search in such a space efficiently. The algorithm is further sped up by a filter and a predictor, which can avoid repeatedly training SFs with same expressive ability and help removing bad candidates during the search before model training. Finally, we perform extensive experiments on benchmark data sets. Results on link prediction and triplets classification show that the searched SFs by AutoSF, are KG dependent, new to the literature, and outperform the state-of- the-art SFs designed by humans. 1

Authors

Quanming Yao

4 papers

Yongqi Zhang

3 papers

Lei Chen

3 papers

TL;DR

Abstract

Authors

References70 items

Anytime Bottom-Up Rule Learning for Knowledge Graph Completion

Efficient Neural Interaction Function Search for Collaborative Filtering

Searching for Interaction Functions in Collaborative Filtering

Efficient Neural Architecture Search via Proximal Iterations

Differentiable Neural Architecture Search via Proximal Iterations

Quaternion Knowledge Graph Embeddings

TuckER: Tensor Factorization for Knowledge Graph Completion

Knowledge Representation Learning: A Quantitative Review

NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding

Automated Machine Learning: From Principles to Practices

On Evaluating Embedding Models for Knowledge Base Completion

RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Neural Architecture Optimization

BOHB: Robust and Efficient Hyperparameter Optimization at Scale

DARTS: Differentiable Architecture Search

Canonical Tensor Decomposition for Knowledge Base Completion

SimplE Embedding for Link Prediction in Knowledge Graphs

Progressive Neural Architecture Search

Knowledge Graph Embedding: A Survey of Approaches and Applications

Automatic differentiation in PyTorch

On Multi-Relational Link Prediction with Bilinear Models

Convolutional 2D Knowledge Graph Embeddings

Analogical Inference for Multi-relational Embeddings

Neural Network-based Question Answering over Knowledge Graphs on Word and Character Level

Genetic CNN

Differentiable Learning of Logical Rules for Knowledge Base Reasoning

Knowledge Graph Completion via Complex Tensor Factorization

On the Equivalence of Holographic and Complex Embeddings for Link Prediction

Knowledge graph refinement: A survey of approaches and evaluation methods

Designing Neural Network Architectures using Reinforcement Learning

Neural Architecture Search with Reinforcement Learning

Collaborative Knowledge Base Embedding for Recommender Systems

AMiner: Mining Deep Knowledge from Big Scholar Data

Efficient and Robust Automated Machine Learning

Holographic Embeddings of Knowledge Graphs

Observed versus latent features for knowledge base and text inference

Knowledge Graph Embedding via Dynamic Mapping Matrix

A Review of Relational Machine Learning for Knowledge Graphs

Learning Entity and Relation Embeddings for Knowledge Graph Completion

Efficient Benchmarking of Hyperparameter Optimizers via Surrogates

Adam: A Method for Stochastic Optimization

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

Transition-based Knowledge Graph Embedding with Relational Mapping Properties

Knowledge vault: a web-scale approach to probabilistic knowledge fusion

Knowledge Graph Embedding by Translating on Hyperplanes

Translating Embeddings for Modeling Multi-relational Data

Reasoning With Neural Tensor Networks for Knowledge Base Completion

Representation Learning: A Review and New Perspectives

Algorithms for Hyper-Parameter Optimization

Random Walk Inference and Learning in A Large Scale Knowledge Base

A Three-Way Model for Collective Learning on Multi-Relational Data

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

Freebase: a collaboratively created graph database for structuring human knowledge

YAGO: A Core of Semantic Knowledge Unifying WordNet and Wikipedia

Greed is good: algorithmic results for sparse approximation

WordNet: A Lexical Database for English

Jan H

Quaternion Knowledge Graph Embedding

Automated Machine Learning - Methods, Systems, Challenges

Expanding Holographic Embeddings for Knowledge Completion

editors

Deep Learning

YAGO3: A Knowledge Base from Multilingual Wikipedias

Introducing the knowledge graph: Things, not strings

Introduction to Statistical Relational Learning

Approximation with artificial neural networks

we propose a progressive greedy algorithm based on the inductive rule (7), which can signiﬁcantly cut down

Learn from small samples : as the real performance of each point in the search space is expensive to acquire, the complexity of the predictor should be low so that it can learn from few samples

Correlate well with true performance : the predictor needs not to accurately predict the exact values of validation performance, instead it should rank good candidates over bad ones

(C1). g ( r ) can be symmetric with proper r and skew-symmetric with proper r (cid:48)

Field of Study

Journal Information

Name

Page

Venue Information

Name