Graph Representation Learning via Aggregation Enhancement (2022-01-30T00:00:00.000000Z)

TL;DR

This work highlights the potential of KR to advance the field of graph representation learning and enhance the performance of GNNs, using KR loss as the primary loss in self-supervised settings or as a regularization term in supervised settings.

Abstract

Graph neural networks (GNNs) have become a powerful tool for processing graph-structured data but still face challenges in effectively aggregating and propagating information between layers, which limits their performance. We tackle this problem with the kernel regression (KR) approach, using KR loss as the primary loss in self-supervised settings or as a regularization term in supervised settings. We show substantial performance improvements compared to state-of-the-art in both scenarios on multiple transductive and inductive node classification datasets, especially for deep networks. As opposed to mutual information (MI), KR loss is convex and easy to estimate in high-dimensional cases, even though it indirectly maximizes the MI between its inputs. Our work highlights the potential of KR to advance the field of graph representation learning and enhance the performance of GNNs. The code to reproduce our experiments is available at https://github.com/Anonymous1252022/KR_for_GNNs

Authors

Chaim Baskin

5 papers

Evgenii Zheltonozhskii

2 papers

Maxim Fishman

1 papers

TL;DR

Abstract

Authors

References90 items

Gradient Gating for Deep Multi-Rate Learning on Graphs

GraphMAE: Self-Supervised Masked Graph Autoencoders

How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

Unsupervised Network Embedding Beyond Homophily

SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation

Bayesian Graph Contrastive Learning

Understanding over-squashing and bottlenecks on graphs via curvature

Toward Graph Self-Supervised Learning With Contrastive Adjusted Zooming

Multi-View Attributed Graph Clustering

Graph Contrastive Learning Automated

Self-supervised Graph-level Representation Learning with Local and Global Structure

How Attentive are Graph Attention Networks?

A connectomic study of a petascale fragment of human cerebral cortex

Maximizing Mutual Information Across Feature and Topology Views for Learning Graph Representations

Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning

A computational framework for modeling complex sensor network data using graph signal processing and graph neural networks in structural health monitoring

Node Embedding using Mutual Information and Self-Supervision based Bi-level Aggregation

Graph Self-Supervised Learning: A Survey

Pre-Training on Dynamic Graph Neural Networks

Structured Graph Learning for Scalable Subspace Clustering: From Single View to Multiview

HDMI: High-order Deep Multiplex Infomax

Large-Scale Representation Learning on Graphs via Bootstrapping

Node Similarity Preserving Graph Convolutional Networks

Iterative Graph Self-Distillation

Graph Contrastive Learning with Augmentations

Usable Information and Evolution of Optimal Representations During Training

Graph-Based Neural Network Models with Multiple Self-Supervised Auxiliary Tasks

Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning

Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning

Contrastive Self-supervised Learning for Graph Classification

CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning

The linear conditional expectation in Hilbert space

TUDataset: A collection of benchmark datasets for learning with graphs

Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks

GPT-GNN: Generative Pre-Training of Graph Neural Networks

Self-supervised Learning on Graphs: Deep Insights and New Direction

On the Bottleneck of Graph Neural Networks and its Practical Implications

Unsupervised Graph Representation by Periphery and Hierarchical Information Maximization

Deep Graph Contrastive Representation Learning

Self-supervised Training of Graph Convolutional Networks

Contrastive Multi-View Representation Learning on Graphs

Open Graph Benchmark: Datasets for Machine Learning on Graphs

Self-Supervised Graph Representation Learning via Global Context Prediction

Geom-GCN: Geometric Graph Convolutional Networks

Supervised Learning on Relational Databases with Graph Neural Networks

MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding

Graph Representation Learning via Graphical Mutual Information Maximization

Diffusion Improves Graph Learning

Semi-Implicit Graph Variational Auto-Encoders

Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning

InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization

GraphSAINT: Graph Sampling Based Inductive Learning Method

Pre-Training Graph Neural Networks for Generic Structural Feature Extraction

Neural Entropic Estimation: A faster path to mutual information estimation

Strategies for Pre-training Graph Neural Networks

Spatio-Temporal Deep Graph Infomax

Active 3-D Shape Cosegmentation With Graph Convolutional Networks

Pitfalls of Graph Neural Network Evaluation

Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks

Deep Graph Infomax

MINE: Mutual Information Neural Estimation

Adversarially Regularized Graph Autoencoder

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

"Like Sheep Among Wolves": Characterizing Hateful Users on Twitter

MGAE: Marginalized Graph Autoencoder for Graph Clustering

Fixing a Broken ELBO

Graph Attention Networks

Predicting multicellular function through multi-layer tissue networks

Attention is All you Need

Inductive Representation Learning on Large Graphs

struc2vec: Learning Node Representations from Structural Identity

Neural Message Passing for Quantum Chemistry

Semi-Supervised Classification with Graph Convolutional Networks

Kernel Mean Embedding of Distributions: A Review and Beyonds

Revisiting Semi-Supervised Learning with Graph Embeddings

Variational Inference: A Review for Statisticians