Visual Pivoting for (Unsupervised) Entity Alignment (2020-09-28T00:00:00.000000Z)

TL;DR

This work shows that images are particularly useful to align long-tail KG entities, which inherently lack the structural contexts necessary for capturing the correspondences, and provides EVA, a completely unsupervised solution to this problem.

Abstract

This work studies the use of visual semantic representations to align entities in heterogeneous knowledge graphs (KGs). Images are natural components of many existing KGs. By combining visual knowledge with other auxiliary information, we show that the proposed new approach, EVA, creates a holistic entity representation that provides strong signals for cross-graph entity alignment. Besides, previous entity alignment methods require human labelled seed alignment, restricting availability. EVA provides a completely unsupervised solution by leveraging the visual similarity of entities to create an initial seed dictionary (visual pivots). Experiments on benchmark data sets DBP15k and DWY15k show that EVA offers state-of-the-art performance on both monolingual and cross-lingual entity alignment tasks. Furthermore, we discover that images are particularly useful to align long-tail KG entities, which inherently lack the structural contexts necessary for capturing the correspondences. Code release: https://github.com/cambridgeltl/eva; project page: http://cogcomp.org/page/publication view/927.

Authors

Dan Roth

19 papers

Nigel Collier

12 papers

Muhao Chen

11 papers

TL;DR

Abstract

Authors

References87 items

Multilingual Knowledge Graph Completion via Ensemble Knowledge Transfer

Exploring and Evaluating Attributes, Values, and Structure for Entity Alignment

Relational Reflection Entity Alignment

COTSAE: CO-Training of Structure and Attribute Embeddings for Entity Alignment

Visual Grounding in Video for Unsupervised Word Translation

A benchmarking study of embedding-based entity alignment for knowledge graphs

Open Knowledge Enrichment for Long-tail Entities

HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubs

Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation

Modeling Multi-mapping Relations for Precise Cross-lingual Entity Alignment

Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model

A Survey on Knowledge Graph Embeddings with Literals: Which model links better Literal-ly?

TransEdge: Translating Relation-Contextualized Embeddings for Knowledge Graphs

Aligning Cross-Lingual Entities with Multi-Aspect Information

Jointly Learning Entity and Relation Representations for Entity Alignment

Improving Cross-lingual Entity Alignment via Optimal Transport

Neighborhood-Aware Attentional Representation for Multilingual Knowledge Graphs

Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts

Entity Alignment between Knowledge Graphs Using Attribute Embeddings

Multi-Channel Graph Neural Network for Entity Alignment

Multi-view Knowledge Graph Embedding for Entity Alignment

Learning to Exploit Long-term Relational Dependencies in Knowledge Graphs

Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs

Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network

Text Generation from Knowledge Graphs with Graph Transformers

MMKG: Multi-Modal Knowledge Graphs

KBQA: Learning Question Answering over QA Corpora and Knowledge Bases

Unsupervised Multi-Modal Neural Machine Translation

Findings of the Third Shared Task on Multimodal Machine Translation

Embedding Multimodal Relational Data for Knowledge Base Completion

One-Shot Relational Learning for Knowledge Graphs

Cross-lingual Knowledge Projection Using Machine Translation and Target-side Knowledge Base Completion

Bootstrapping Entity Alignment with Knowledge Graph Embedding

Bridging Languages through Images with Deep Partial Canonical Correlation Analysis

Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search

Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment

ELDEN: Improved Entity Linking Using Densified Knowledge Graphs

Places: A 10 Million Image Database for Scene Recognition

A Multimodal Translation-Based Approach for Knowledge Graph Representation Learning

Quantifying the Visual Concreteness of Words and Topics in Multimodal Datasets

Dynamic Meta-Embeddings for Improved Sentence Representations

Decoupled Weight Decay Regularization

Hike: A Hybrid Human-Machine Method for Entity Alignment in Large-Scale Knowledge Bases

Towards Holistic Concept Representations: Embedding Relational Knowledge, Visual Attributes, and Distributional Word Semantics

Word Translation Without Parallel Data

Answering Visual-Relational Queries in Web-Extracted Knowledge Graphs

Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding

Image Pivoting for Learning Multilingual Multimodal Representations

Incorporating Global Visual Features into Attention-based Neural Machine Translation.

Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment

YAGO: A Multilingual Knowledge Base from Wikipedia, Wordnet, and Geonames

Image-embodied Knowledge Representation Learning

Semi-Supervised Classification with Graph Convolutional Networks

Densely Connected Convolutional Networks

Multi-Modal Representations for Improved Bilingual Lexicon Learning

A Shared Task on Multimodal Machine Translation and Crosslingual Image Description

Attention-based Multimodal Neural Machine Translation

Enriching Word Vectors with Subword Information

Does Multimodality Help Human and Machine for Translation and Image Captioning?

Multimodal Pivots for Image Caption Translation

Deep Residual Learning for Image Recognition

Rethinking the Inception Architecture for Computer Vision

Visual Bilingual Lexicon Induction with Transferred ConvNet Features

Learning Image Embeddings using Convolutional Neural Networks for Improved Multi-Modal Semantics

Going deeper with convolutions

Question Answering with Subgraph Embeddings

Improving Multi-Modal Representations Using Image Dispersion: Why Less is Sometimes More

Learning conflict resolution strategies for cross-language Wikipedia data fusion

Translating Embeddings for Modeling Multi-relational Data

PIDGIN: ontology alignment using web text as interlingua

PARIS: Probabilistic Alignment of Relations, Instances, and Schema

Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations

Hubs in Space: Popular Nearest Neighbors in High-Dimensional Data

ImageNet: A large-scale hierarchical image database

Learning to Connect Language and Perception

Freebase: a collaboratively created graph database for structuring human knowledge