Learning Discrete Structures for Graph Neural Networks (2019-03-28T00:00:00.000000Z)

TL;DR

This work proposes to jointly learn the graph structure and the parameters of graph convolutional networks (GCNs) by approximately solving a bilevel program that learns a discrete probability distribution on the edges of the graph.

Abstract

Graph neural networks (GNNs) are a popular class of machine learning models whose major advantage is their ability to incorporate a sparse and discrete dependency structure between data points. Unfortunately, GNNs can only be used when such a graph-structure is available. In practice, however, real-world graphs are often noisy and incomplete or might not be available at all. With this work, we propose to jointly learn the graph structure and the parameters of graph convolutional networks (GCNs) by approximately solving a bilevel program that learns a discrete probability distribution on the edges of the graph. This allows one to apply GCNs not only in scenarios where the given graph is incomplete or corrupted but also in those where a graph is not available. We conduct a series of experiments that analyze the behavior of the proposed method and demonstrate that it outperforms related methods by a significant margin.

Authors

Mathias Niepert

8 papers

M. Pontil

3 papers

Luca Franceschi

2 papers

TL;DR

Abstract

Authors

References63 items

Monte Carlo Gradient Estimation in Machine Learning

Bilevel Programming for Hyperparameter Optimization and Meta-Learning

MolGAN: An implicit generative model for small molecular graphs

Graphite: Iterative Generative Modeling of Graphs

Link Prediction Based on Graph Neural Networks

GraphRNN: A Deep Generative Model for Graphs

Learning Deep Generative Models of Graphs

Neural Relational Inference for Interacting Systems

Backpropagation through the Void: Optimizing control variates for black-box gradient estimation

Graph Attention Networks

Learning Graph Representations with Embedding Propagation

Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization

Weisfeiler-Lehman Neural Machine for Link Prediction

Inductive Representation Learning on Large Graphs

Unbiasing Truncated Backpropagation Through Time

Neural Message Passing for Quantum Chemistry

Forward and Reverse Gradient-Based Hyperparameter Optimization

REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models

FMA: A Dataset for Music Analysis

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs

Categorical Reparameterization with Gumbel-Softmax

The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables

Learning Graphical State Transitions

Semi-Supervised Classification with Graph Convolutional Networks

Revisiting Semi-Supervised Learning with Graph Embeddings

Gradient Estimation Using Stochastic Computation Graphs

A Review of Relational Machine Learning for Knowledge Graphs

Gradient-based Hyperparameter Optimization through Reversible Learning

Adam: A Method for Stochastic Optimization

Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

An Introduction to Statistical Learning

Generic Methods for Optimization-Based Modeling

Scikit-learn: Machine Learning in Python

Link Prediction in Complex Networks: A Survey

Kronecker Graphs: An Approach to Modeling Networks

Collective Classification in Network Data

Deep learning via semi-supervised embedding

An overview of bilevel optimization

Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples

Model Selection via Bilevel Optimization

Graphs over time: densification laws, shrinking diameters and possible explanations

The link prediction problem for social networks

Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions

Nonlinear dimensionality reduction by locally linear embedding.

A global geometric framework for nonlinear dimensionality reduction.

Collective dynamics of ‘small-world’ networks

An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories

Backpropagation Through Time: What It Does and How to Do It

Bilevel learning of the Group Lasso structure

Hyperparameter Optimization

TensorFlow: Large-scale machine learning on heterogeneous systems

Dropout: a simple way to prevent neural networks from overfitting

Learning constrained task similarities in graph-regularized multitask learning. Regularization, Optimization, Kernels, and Support Vector Machines

Large-Scale Machine Learning with Stochastic Gradient Descent

The Graph Neural Network Model

Visualizing Data using t-SNE

Collective Classi!cation in Network Data

Introduction to Statistical Relational Learning

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition

An E cient Gradient-Based Algorithm for On-LineTraining of Recurrent Network Trajectories

On the evolution of random graphs

Recall that F can be computed only after (approximately) solving the inner optimization problem

Field of Study

Venue Information

Name

Type

URL

Alternate Names