Graph Self-supervised Learning with Accurate Discrepancy Learning (2022-02-07T00:00:00.000000Z)

TL;DR

This work proposes a framework that aims to learn the exact discrepancy between the original and the perturbed graphs, coined as Discrepancy-based Self-supervised LeArning (D-SLA), and validate it on various graph-related downstream tasks, on which the authors' largely outperforms relevant baselines.

Abstract

Self-supervised learning of graph neural networks (GNNs) aims to learn an accurate representation of the graphs in an unsupervised manner, to obtain transferable representations of them for diverse downstream tasks. Predictive learning and contrastive learning are the two most prevalent approaches for graph self-supervised learning. However, they have their own drawbacks. While the predictive learning methods can learn the contextual relationships between neighboring nodes and edges, they cannot learn global graph-level similarities. Contrastive learning, while it can learn global graph-level similarities, its objective to maximize the similarity between two differently perturbed graphs may result in representations that cannot discriminate two similar graphs with different properties. To tackle such limitations, we propose a framework that aims to learn the exact discrepancy between the original and the perturbed graphs, coined as Discrepancy-based Self-supervised LeArning (D-SLA). Specifically, we create multiple perturbations of the given graph with varying degrees of similarity, and train the model to predict whether each graph is the original graph or the perturbed one. Moreover, we further aim to accurately capture the amount of discrepancy for each perturbed graph using the graph edit distance. We validate our D-SLA on various graph-related downstream tasks, including molecular property prediction, protein function prediction, and link prediction tasks, on which ours largely outperforms relevant baselines.

Authors

Sung Ju Hwang

8 papers

Jinheon Baek

4 papers

Dongki Kim

2 papers

TL;DR

Abstract

Authors

References72 items

How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation

Are Graph Augmentations Necessary?: Simple Graph Contrastive Learning for Recommendation

InfoGCL: Information-Aware Graph Contrastive Learning

Graph Contrastive Learning Automated

Adversarial Graph Augmentation to Improve Graph Contrastive Learning

Self-supervised Graph-level Representation Learning with Local and Global Structure

MARS: Markov Molecular Sampling for Multi-objective Drug Discovery

Accurate Learning of Graph Representations with Graph Multiset Pooling

Large-Scale Representation Learning on Graphs via Bootstrapping

SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism

Biological network analysis with deep learning

Graph Contrastive Learning with Adaptive Augmentation

Graph Contrastive Learning with Augmentations

TUDataset: A collection of benchmark datasets for learning with graphs

Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs

GPT-GNN: Generative Pre-Training of Graph Neural Networks

Self-Supervised Graph Transformer on Large-Scale Molecular Data

Learning to Extrapolate Knowledge: Transductive Few-shot Out-of-Graph Link Prediction

Open Graph Benchmark: Datasets for Machine Learning on Graphs

A Simple Framework for Contrastive Learning of Visual Representations

Graph-Bert: Only Attention is Needed for Learning Graph Representations

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Momentum Contrast for Unsupervised Visual Representation Learning

InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization

Strategies for Pre-training Graph Neural Networks

Graph U-Nets

Graph Matching Networks for Learning the Similarity of Graph Structured Objects

Self-Attention Graph Pooling

Fast Graph Representation Learning with PyTorch Geometric

Graph Neural Networks for Social Recommendation

Evolution of resilience in protein interactomes across the tree of life

How Powerful are Graph Neural Networks?

Deep Graph Infomax

Learning-Based Efficient Graph Similarity Computation via Multi-Scale Convolutional Set Matching

Hierarchical Graph Representation Learning with Differentiable Pooling

Unsupervised Feature Learning via Non-parametric Instance Discrimination

Graph Attention Networks

Inductive Representation Learning on Large Graphs

Neural Message Passing for Quantum Chemistry

MoleculeNet: a benchmark for molecular machine learning

Semi-Supervised Classification with Graph Convolutional Networks

Principal component analysis: a review and recent developments

ZINC 15 – Ligand Discovery for Everyone

FaceNet: A unified embedding for face recognition and clustering

Weisfeiler-Lehman Graph Kernels

Extended-Connectivity Fingerprints

Comparing Stars: On Approximating Graph Edit Distance

A distance measure between attributed relational graphs for pattern recognition

Graph Adversarial Self-Supervised Learning

COLLAB dataset consists of ego-networks extracted from public scientific collaboration networks, namely High Energy Physics

Implementation Details on Protein Function Prediction We use the pre-defined biological graphs

The Evolution of Resilience

Visualizing Data using t-SNE

A reduction of a graph to a canonical form and an algebra arising during this reduction

is a predictive learning baseline that predicts the attributes of masked nodes and edges from the embeddings of nodes

is a variant of JOAO, which has individual projection heads according to the perturbation schemes. Specifically

Link Prediction Table 6. Statistics of social network datasets used in link prediction experiments

If you are using existing assets (

(a) Did you state the full set of assumptions of all theoretical results

Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We provide the type of GPUs in Section

Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Did you discuss any potential negative societal impacts of your work? [Yes] We discuss the potential societal impacts in Section 6

Have you read the ethics review guidelines and ensured that your paper conforms to them

Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We provide the detailed experimental setups in section A of Appendix

GraphLoG

Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation

EdgePred is a predictive learning baseline adopted from the link prediction task of Hamilton et al. [2], whose goal is to predict the existence of edges between the given two nodes

Did you include any new assets either in the supplemental material or as a URL?

c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] All of our main results contain variances or standard deviations of multiple runs

a contrastive learning baseline that, while the learning objective of it is the same as the GraphCL model described above, learns to automatically select the perturbation schemes

Field of Study

Journal Information

Name

Volume