MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid (2022-12-29T00:00:00.000000Z)

TL;DR

MEAformer is introduced, a mlti-modal entity alignment transformer approach for meta modality hybrid, which dynamically predicts the mutual correlation coefficients among modalities for more fine-grained entity-level modality fusion and alignment.

Abstract

Multi-modal entity alignment (MMEA) aims to discover identical entities across different knowledge graphs (KGs) whose entities are associated with relevant images. However, current MMEA algorithms rely on KG-level modality fusion strategies for multi-modal entity representation, which ignores the variations of modality preferences of different entities, thus compromising robustness against noise in modalities such as blurry images and relations. This paper introduces MEAformer, a mlti-modal entity alignment transformer approach for meta modality hybrid, which dynamically predicts the mutual correlation coefficients among modalities for more fine-grained entity-level modality fusion and alignment. Experimental results demonstrate that our model not only achieves SOTA performance in multiple training scenarios, including supervised, unsupervised, iterative, and low-resource settings, but also has a limited number of parameters, efficient runtime, and interpretability. Our code is available at https://github.com/zjukg/MEAformer.

Authors

Hua-zeng Chen

10 papers

Zhuo Chen

10 papers

Lingbing Guo

6 papers

TL;DR

Abstract

Authors

References69 items

Revisiting Embedding-Based Entity Alignment: A Robust and Adaptive Method

Knowledge graph-enhanced molecular contrastive learning with functional prompt

Vision-Language Pre-training: Basics, Recent Advances, and Future Trends

Leveraging Multi-Modal Information for Cross-Lingual Entity Matching across Knowledge Graphs

Multi-modal Contrastive Representation Learning for Entity Alignment

Multi-modal Siamese Network for Entity Alignment

LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injection

ClusterEA: Scalable Entity Alignment with Stochastic Training and Normalized Mini-batch Similarities

Entity Alignment with Reliable Path Reasoning and Relation-Aware Heterogeneous Graph Transformer

Semantics Driven Embedding Learning for Effective Entity Alignment

Graph Neural Networks for Recommender System

Informed Multi-context Entity Alignment

Molecular Contrastive Learning with Chemical Element Knowledge Graph

Entity and Relation Matching Consensus for Entity Alignment

From Alignment to Assignment: Frustratingly Simple Unsupervised Entity Alignment

Zero-shot Visual Question Answering using Knowledge Graph

Make It Easy: An Effective End-to-End Entity Alignment Framework

OntoEA: Ontology-guided Entity Alignment via Joint Knowledge Graph Embedding

A benchmark and comprehensive survey on knowledge graph entity alignment via representation learning

DeepViT: Towards Deeper Vision Transformer

OntoZSL: Ontology-enhanced Zero-shot Learning

Relation-Aware Neighborhood Matching Model for Entity Alignment

Exploring and Evaluating Attributes, Values, and Structure for Entity Alignment

Visual Pivoting for (Unsupervised) Entity Alignment

MMEA: Entity Alignment for Multi-modal Knowledge Graph

BERT-INT: A BERT-based Interaction Model For Knowledge Graph Alignment

Neighborhood Matching Network for Entity Alignment

Meta-Learning in Neural Networks: A Survey

A Simple Framework for Contrastive Learning of Visual Representations

MRAEA: An Efficient and Robust Entity Alignment Approach for Cross-lingual Knowledge Graph

Collective Entity Alignment via Adaptive Features

Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation

Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model

TransEdge: Translating Relation-Contextualized Embeddings for Knowledge Graphs

Aligning Cross-Lingual Entities with Multi-Aspect Information

Neighborhood-Aware Attentional Representation for Multilingual Knowledge Graphs

Entity Alignment between Knowledge Graphs Using Attribute Embeddings

Multi-Channel Graph Neural Network for Entity Alignment

Multi-view Knowledge Graph Embedding for Entity Alignment

Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs

Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network

MMKG: Multi-Modal Knowledge Graphs

Bootstrapping Entity Alignment with Knowledge Graph Embedding

Graph Attention Networks

Iterative Entity Alignment via Joint Knowledge Embeddings

Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding

Attention is All you Need

Semi-Supervised Classification with Graph Convolutional Networks

Deep Residual Learning for Image Recognition

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

Very Deep Convolutional Networks for Large-Scale Image Recognition

Translating Embeddings for Modeling Multi-relational Data

PARIS: Probabilistic Alignment of Relations, Instances, and Schema

LogMap: Logic-Based and Scalable Ontology Matching

volume 12922 of Lecture Notes in Computer Science

volume 119 of Proceedings of Machine Learning Research

pages 6477–6487

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Pan , Zonggang Yuan , and Huajun Chen . 2021 . Zero - Shot Visual Question Answering Using Knowledge Graph . In ISWC ( Lecture Notes in Computer Science , Vol . 12922 )

In EMNLP/IJCNLP (1)

pages 1943–1949

In ICLR (Poster)

In ISWC (1)

pages 4258–4264

In NIPS

Alberto Garcı́a-Durán

applied sciences

Values

MM ’23, October 29-November 3, 2023, Ottawa, ON, Canada

Field of Study

Journal Information

Name

Venue Information

Name

Type

URL