InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization (2019-07-31T00:00:00.000000Z)

TL;DR

Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state- of- the-art semi-supervised models.

Abstract

This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models.

Authors

References75 items

Utilizing Edge Features in Graph Neural Networks via Variational Information Maximization

Strategies for Pre-training Graph Neural Networks

Pre-training Graph Neural Networks

Interpolation Consistency Training for Semi-Supervised Learning

Fast Graph Representation Learning with PyTorch Geometric

Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals

How Powerful are Graph Neural Networks?

Deep Graph Infomax

Learning deep representations by mutual information estimation and maximization

Hierarchical Graph Representation Learning with Differentiable Pooling

Manifold Mixup: Better Representations by Interpolating Hidden States

Representation Learning on Graphs with Jumping Knowledge Networks

Sub2Vec: Feature Learning for Subgraphs

Anonymous Walk Embeddings

An End-to-End Deep Learning Architecture for Graph Classification

Disentangling by Factorising

Neural Relational Inference for Interacting Systems

Junction Tree Variational Autoencoder for Molecular Graph Generation

Semi-supervised learning of hierarchical representations of molecules using neural message passing

Automatic differentiation in PyTorch

Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties.

Adversarial Examples for Evaluating Reading Comprehension Systems

graph2vec: Learning Distributed Representations of Graphs

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

Neural Message Passing for Quantum Chemistry

Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results

Detecting subnetwork-level dynamic correlations

Temporal Ensembling for Semi-Supervised Learning

Semi-Supervised Classification with Graph Convolutional Networks

node2vec: Scalable Feature Learning for Networks

Mutual exclusivity loss for semi-supervised deep learning

On Valid Optimal Assignment Kernels and Applications to Graph Classification

f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization

Learning Convolutional Neural Networks for Graphs

The Multiscale Laplacian Graph Kernel

Convolutional Networks on Graphs for Learning Molecular Fingerprints

Deep Graph Kernels

Graph Invariant Kernels

LINE: Large-scale Information Network Embedding

Adam: A Method for Stochastic Optimization

Quantum chemistry structures and properties of 134 kilo molecules

Distributed Representations of Sentences and Documents

EgoNet: identification of human disease ego-network modules

DeepWalk: online learning of social representations

Auto-Encoding Variational Bayes

Learning word embeddings efficiently with noise-contrastive estimation

Scalable kernels for graphs with continuous attributes

Distributed Representations of Words and Phrases and their Compositionality

A Fast Approximation of the Weisfeiler-Lehman Graph Kernel for RDF Data

Efficient Estimation of Word Representations in Vector Space

Efficient Graph Kernels by Randomization

Temporal link prediction by integrating content and structure information

Community discovery using nonnegative matrix factorization

Using graph theory to analyze biological networks

LIBSVM: A library for support vector machines

Weisfeiler-Lehman Graph Kernels

A Survey on Transfer Learning

Efficient graphlet kernels for large graph comparison

A unified architecture for natural language processing: deep neural networks with multitask learning

Biological network comparison using graphlet degree distribution

Semi-Supervised Learning

Global landscape of protein complexes in the yeast Saccharomyces cerevisiae

Shortest-path kernels on graphs

Semi-supervised Learning by Entropy Minimization

Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions

Marginalized Kernels Between Labeled Graphs

Finding and evaluating community structure in networks.

Realistic Evaluation of Semi-Supervised Learning Algorithms

Hunt For The Unique, Stable, Sparse And Fast Feature Learning On Graphs

Domain adaptation and sample bias correction theory and algorithm for regression

Latent Dirichlet Allocation

To transfer or not to transfer

On Graph Kernels: Hardness Results and Efficient Alternatives

A reduction of a graph to a canonical form and an algebra arising during this reduction

QM9 a description of the properties in the QM9 dataset see section 10.2 of [15]