Message Passing Attention Networks for Document Understanding (2019-08-17T00:00:00.000000Z)

TL;DR

This paper represents documents as word co-occurrence networks and proposes an application of the message passing framework to NLP, the Message Passing Attention network for Document understanding (MPAD), and proposes several hierarchical variants of MPAD.

Abstract

Graph neural networks have recently emerged as a very effective framework for processing graph-structured data. These models have achieved state-of-the-art performance in many tasks. Most graph neural networks can be described in terms of message passing, vertex update, and readout functions. In this paper, we represent documents as word co-occurrence networks and propose an application of the message passing framework to NLP, the Message Passing Attention network for Document understanding (MPAD). We also propose several hierarchical variants of MPAD. Experiments conducted on 10 standard text classification datasets show that our architectures are competitive with the state-of-the-art. Ablation studies reveal further insights about the impact of the different components on performance. Code is publicly available at: this https URL .

Authors

A. Tixier

6 papers

M. Vazirgiannis

8 papers

Giannis Nikolentzos

2 papers

TL;DR

Abstract

Authors

References66 items

Convolutional Networks

Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding

Dialogue Act Classification with Context-Aware Self-Attention

How Powerful are Graph Neural Networks?

Graph Convolutional Networks for Text Classification

Representation Learning on Graphs with Jumping Knowledge Networks

Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN

Graph Attention Networks

Kernel Graph Convolutional Neural Networks

Automatic differentiation in PyTorch

DiSAN: Directional Self-Attention Network for RNN/CNN-free Language Understanding

Shortest-Path Graph Kernels for Document Similarity

Graph Classification with 2D Convolutional Neural Networks

Attention is All you Need

Inductive Representation Learning on Large Graphs

Neural Message Passing for Quantum Chemistry

A Structured Self-attentive Sentence Embedding

Supervised Word Mover's Distance

Interaction Networks for Learning about Objects, Relations and Physics

Semi-Supervised Classification with Graph Convolutional Networks

GoWvis: A Web Application for Graph-of-Words-based Text Visualization and Summarization

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Hierarchical Attention Networks for Document Classification

Learning Convolutional Neural Networks for Graphs

Molecular graph convolutions: moving beyond fingerprints

Long Short-Term Memory-Networks for Machine Reading

Deep Residual Learning for Image Recognition

A C-LSTM Neural Network for Text Classification

Gated Graph Sequence Neural Networks

Diffusion-Convolutional Neural Networks

Convolutional Networks on Graphs for Learning Molecular Fingerprints

Document Modeling with Gated Recurrent Neural Network for Sentiment Classification

Hierarchical Recurrent Neural Network for Document Modeling

From Word Embeddings To Document Distances

Semantic Clustering and Convolutional Neural Network for Short Text Categorization

Deep Unordered Composition Rivals Syntactic Methods for Text Classification

Deep Convolutional Networks on Graph-Structured Data

A Hierarchical Neural Autoencoder for Paragraphs and Documents

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Adam: A Method for Stochastic Optimization

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Deep Recursive Neural Networks for Compositionality in Language

Convolutional Neural Networks for Sentence Classification

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Distributed Representations of Sentences and Documents

Distributed Representations of Words and Phrases and their Compositionality

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Learning Word Vectors for Sentiment Analysis

Software Framework for Topic Modelling with Large Corpora

Practical solutions to the problem of diagonal dominance in kernel document clustering

A new model for learning in graph domains

Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts

TextRank: Bringing Order into Text

Learning Question Classifiers

Loopy Belief Propagation for Approximate Inference: An Empirical Study

Graph Classification with 2 D Convolutional Neural Networks

and Welling

Dropout: a simple way to prevent neural networks from overfitting

and Mikolov

and Sojka

The Graph Neural Network Model

Annotating Expressions of Opinions and Emotions in Language

and Lee

and Tarau

On reduction

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names