DeeperGCN: All You Need to Train Deeper GCNs (2020-06-13T00:00:00.000000Z)

TL;DR

Extensive experiments on Open Graph Benchmark show DeeperGCN significantly boosts performance over the state-of-the-art on the large scale graph learning tasks of node property prediction and graph property prediction.

Abstract

Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper. These challenges limit the representation power of GCNs on large-scale graphs. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs. We define differentiable generalized aggregation functions to unify different message aggregation operations (e.g. mean, max). We also propose a novel normalization layer namely MsgNorm and a pre-activation version of residual connections for GCNs. Extensive experiments on Open Graph Benchmark (OGB) show DeeperGCN significantly boosts performance over the state-of-the-art on the large scale graph learning tasks of node property prediction and graph property prediction. Please visit this https URL for more information.

Authors

Bernard Ghanem

28 papers

Ali K. Thabet

2 papers

Guohao Li

2 papers

TL;DR

Abstract

Authors

References59 items

Open Graph Benchmark: Datasets for Machine Learning on Graphs

PairNorm: Tackling Oversmoothing in GNNs

DropEdge: Towards Deep Graph Convolutional Networks on Node Classification

GraphSAINT: Graph Sampling Based Inductive Learning Method

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Universal Invariant and Equivariant Graph Neural Networks

Graph Wavelet Neural Network

DeepGCNs: Can GCNs Go As Deep As CNNs?

Fast Graph Representation Learning with PyTorch Geometric

On the Universality of Invariant Networks

How Powerful are Graph Neural Networks?

Invariant and Equivariant Graph Networks

Predict then Propagate: Graph Neural Networks meet Personalized PageRank

Large-Scale Learnable Graph Convolutional Networks

Representation Learning on Graphs with Jumping Knowledge Networks

Graph Convolutional Neural Networks for Web-Scale Recommender Systems

Relational inductive biases, deep learning, and graph networks

Semi-supervised User Geolocation via Graph Convolutional Networks

GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs

An Experimental Study of Neural Networks for Variable Graphs

FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling

Dynamic Graph CNN for Learning on Point Clouds

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

Adaptive Graph Convolutional Neural Networks

Graph Attention Networks

Stochastic Training of Graph Convolutional Networks with Variance Reduction

Predicting multicellular function through multi-layer tissue networks

Inductive Representation Learning on Large Graphs

CayleyNets: Graph Convolutional Neural Networks With Complex Rational Spectral Filters

Neural Message Passing for Quantum Chemistry

On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning

Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks

Deep Sets

Joint 2D-3D-Semantic Data for Indoor Scene Understanding

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs

Semi-Supervised Classification with Graph Convolutional Networks

Densely Connected Convolutional Networks

Layer Normalization

node2vec: Scalable Feature Learning for Networks

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Learning Convolutional Neural Networks for Graphs

Revisiting Semi-Supervised Learning with Graph Embeddings

Identity Mappings in Deep Residual Networks

Deep Residual Learning for Image Recognition

Multi-Scale Context Aggregation by Dilated Convolutions

Deep Convolutional Networks on Graph-Structured Data

Distilling the Knowledge in a Neural Network

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Neural Machine Translation by Jointly Learning to Align and Translate

DeepWalk: online learning of social representations

Spectral Networks and Locally Connected Networks on Graphs

Rectified Linear Units Improve Restricted Boltzmann Machines

Relational learning via latent social dimensions

Comparison of descriptor spaces for chemical compound retrieval and classification

A Tutorial on Energy-Based Learning

A reduction of a graph to a canonical form and an algebra arising during this reduction

Sur la notion de la moyenne

used for each layer

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names