Simple GNN Regularisation for 3D Molecular Property Prediction&Beyond (2021-06-15T00:00:00.000000Z)

TL;DR

It is shown that simple noise regularisation can be an effective way to address GNN oversmoothing, and Noisy Nodes, a simple technique in which it is derived, can serve as a complementary building block in the GNN toolkit.

Abstract

In this paper we show that simple noise regularisation can be an effective way to address GNN oversmoothing. First we argue that regularisers addressing oversmoothing should both penalise node latent similarity and encourage meaningful node representations. From this observation we derive"Noisy Nodes", a simple technique in which we corrupt the input graph with noise, and add a noise correcting node-level loss. The diverse node level loss encourages latent node diversity, and the denoising objective encourages graph manifold learning. Our regulariser applies well-studied methods in simple, straightforward ways which allow even generic architectures to overcome oversmoothing and achieve state of the art results on quantum chemistry tasks, and improve results significantly on Open Graph Benchmark (OGB) datasets. Our results suggest Noisy Nodes can serve as a complementary building block in the GNN toolkit.

Authors

P. Battaglia

13 papers

Petar Velivckovi'c

4 papers

Alvaro Sanchez-Gonzalez

7 papers

TL;DR

Abstract

Authors

References72 items

Graph

Highly accurate protein structure prediction with AlphaFold

Training Graph Neural Networks with 1000 Layers

Do Transformers Really Perform Bad for Graph Representation?

OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs

ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations

SE(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials

E(n) Equivariant Graph Neural Networks

Spherical Message Passing for 3D Molecular Graphs

E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials

Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules

Graph Contrastive Learning with Augmentations

The Open Catalyst 2020 (OC20) Dataset and Community Challenges

FLAG: Adversarial Data Augmentation for Graph Neural Networks

Learning Mesh-Based Simulation with Graph Networks

Graph Convolutional Neural Networks with Node Transition Probability-based Message Passing and DropNode Regularization

Graph neural networks in particle physics

Building powerful and equivariant graph neural networks with structural message-passing

A Note on Over-Smoothing for Graph Neural Networks

SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks

DeeperGCN: All You Need to Train Deeper GCNs

Effective Training Strategies for Deep Graph Neural Networks

Language Models are Few-Shot Learners

How hard is to distinguish graphs with graph neural networks?

Open Graph Benchmark: Datasets for Machine Learning on Graphs

Revisiting "Over-smoothing" in Deep GCNs

Directional Message Passing for Molecular Graphs

Unveiling the predictive power of static structure in glassy systems

Learning to Simulate Complex Physics with Graph Networks

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

Random Features Strengthen Graph Neural Networks

Coloring graph neural networks for node disambiguation

Equivariant Flows: sampling configurations for multi-body systems with symmetric energies

PairNorm: Tackling Oversmoothing in GNNs

Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View

The Truly Deep Graph Convolutional Networks for Node Classification

Generative Modeling by Estimating Gradients of the Data Distribution

Cormorant: Covariant Molecular Neural Networks

Strategies for Pre-training Graph Neural Networks

DeepGCNs: Can GCNs Go As Deep As CNNs?

Relational Pooling for Graph Representations

PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges.

Graph Neural Networks: A Review of Methods and Applications

How Powerful are Graph Neural Networks?

Graph networks as learnable physics engines for inference and control

Relational inductive biases, deep learning, and graph networks

Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

Covariant Compositional Networks For Learning Graphs

SchNet: A continuous-filter convolutional neural network for modeling quantum interactions

Attention is All you Need

Neural Message Passing for Quantum Chemistry

Geometric Deep Learning: Going beyond Euclidean data

Machine learning of accurate energy-conserving molecular force fields

Semi-Supervised Classification with Graph Convolutional Networks

Convolutional Networks on Graphs for Learning Molecular Fingerprints

Adam: A Method for Stochastic Optimization

Quantum chemistry structures and properties of 134 kilo molecules

A Connection Between Score Matching and Denoising Autoencoders

Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion

Extracting and composing robust features with denoising autoencoders

Creating artificial neural networks that generalize

Bootstrapped Representation Learning on Graphs

E(n) Equivariant Graph Neural Networks

Adaptive Graph Diffusion Networks with Hop-wise Attention

The DeepMind JAX Ecosystem, 2020

Haiku: Sonnet for JAX

A Comprehensive Survey on Graph Neural Networks

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

JAX: composable transformations of Python+NumPy programs

The Graph Neural Network Model

Current address: Microsoft Research,

Field of Study