Simplifying Graph Convolutional Networks (2019-02-19T00:00:00.000000Z)

TL;DR

This paper successively removes nonlinearities and collapsing weight matrices between consecutive layers, and theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier.

Abstract

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier. Notably, our experimental evaluation demonstrates that these simplifications do not negatively impact accuracy in many downstream applications. Moreover, the resulting model scales to larger datasets, is naturally interpretable, and yields up to two orders of magnitude speedup over FastGCN.

Authors

Tianyi Zhang

4 papers

Felix Wu

4 papers

A. Souza

1 papers

TL;DR

Abstract

Authors

References63 items

Label Efficient Semi-Supervised Learning via Graph Filtering

LanczosNet: Multi-Scale Deep Graph Convolutional Networks

Graph Neural Networks: A Review of Methods and Applications

Classifier and Exemplar Synthesis for Zero-Shot Learning

A simple yet effective baseline for non-attribute graph classification

How Powerful are Graph Neural Networks?

Deep Graph Infomax

Predict then Propagate: Graph Neural Networks meet Personalized PageRank

Graph Convolution over Pruned Dependency Trees Improves Relation Extraction

Graph Convolutional Networks for Text Classification

Attention Models in Graphs

Relational inductive biases, deep learning, and graph networks

Rethinking Knowledge Graph Propagation for Zero-Shot Learning

An End-to-End Deep Learning Architecture for Graph Classification

Semi-supervised User Geolocation via Graph Convolutional Networks

Bootstrapped Graph Diffusions

Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs

GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs

N-GCN: Multi-scale Graph Convolution for Semi-supervised Node Classification

Attention-based Graph Neural Network for Semi-supervised Learning

FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

Graph Attention Networks

Learning Graph Representations with Embedding Propagation

Position-aware Attention and Supervised Data Improve Slot Filling

Inductive Representation Learning on Large Graphs

Bootstrapped Graph Diffusions: Exposing the Power of Nonlinearity

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs

Semi-Supervised Classification with Graph Convolutional Networks

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Deep Networks with Stochastic Depth

Revisiting Semi-Supervised Learning with Graph Embeddings

Diffusion-Convolutional Neural Networks

Deep Graph Kernels

Electronic spectra from TDDFT and machine learning in chemical space.

Adam: A Method for Stochastic Optimization

Hypercolumns for object segmentation and fine-grained localization

DeepWalk: online learning of social representations

Spectral Networks and Locally Connected Networks on Graphs

Geolocation Prediction in Social Media Data by Finding Location Indicative Words

Supervised Text-based Geolocation Using Language Models on an Adaptive Grid

A Latent Variable Model for Geographic Lexical Variation

Collective Classification in Network Data

Deep learning via semi-supervised embedding

Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples

Semi-Supervised Learning on Riemannian Manifolds

Learning with Local and Global Consistency

Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions

Long Short-Term Memory

WordNet: A Lexical Database for English

Backpropagation Applied to Handwritten Zip Code Recognition

On the limited memory BFGS method for large scale optimization

Phoneme recognition using time-delay neural networks

PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS

The perceptron: a probabilistic model for information storage and organization in the brain.

A Comprehensive Survey on Graph Neural Networks

one-hot features for the word and document nodes. In training SGC, we normalize the features to be between 0 and 1 after propagation and train with L-BFGS for 3 steps

Simplifying Graph and spatiotemporal graphs

Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms

Large-Scale Machine Learning with Stochastic Gradient Descent

Collective Classi!cation in Network Data

A Combined Corner and Edge Detector

A 3x3 isotropic gradient operator for image processing

Field of Study

Venue Information

Name

Type

URL

Alternate Names