Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks (2015-02-27T00:00:00.000000Z)

TL;DR

The Tree-LSTM is introduced, a generalization of LSTMs to tree-structured network topologies that outperform all existing systems and strong LSTM baselines on two tasks: predicting the semantic relatedness of two sentences and sentiment classification.

Abstract

Because of their superior ability to preserve sequence information over time, Long Short-Term Memory (LSTM) networks, a type of recurrent neural network with a more complex computational unit, have obtained strong results on a variety of sequence modeling tasks. The only underlying LSTM structure that has been explored so far is a linear chain. However, natural language exhibits syntactic properties that would naturally combine words to phrases. We introduce the Tree-LSTM, a generalization of LSTMs to tree-structured network topologies. Tree-LSTMs outperform all existing systems and strong LSTM baselines on two tasks: predicting the semantic relatedness of two sentences (SemEval 2014, Task 1) and sentiment classification (Stanford Sentiment Treebank).

Authors

R. Socher

30 papers

Christopher D. Manning

24 papers

Kai Sheng Tai

1 papers

TL;DR

Abstract

Authors

References41 items

Deep Recursive Neural Networks for Compositionality in Language

Show and tell: A neural image caption generator

Learning to Execute

GloVe: Global Vectors for Word Representation

Sequence to Sequence Learning with Neural Networks

Neural Machine Translation by Jointly Learning to Align and Translate

Convolutional Neural Networks for Sentence Classification

ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment

SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment

UNAL-NLP: Combining Soft Cardinality Features for Semantic Textual Similarity, Relatedness and Entailment

The Meaning Factory: Formal Semantics for Recognizing Textual Entailment and Determining Semantic Similarity

Illinois-LH: A Denotational and Distributional Approach to Semantics

Distributed Representations of Sentences and Documents

Grounded Compositional Semantics for Finding and Describing Images with Sentences

A Convolutional Neural Network for Modelling Sentences

Hybrid speech recognition with Deep Bidirectional LSTM

Distributed Representations of Words and Phrases and their Compositionality

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Modeling Documents with Deep Boltzmann Machines

PPDB: The Paraphrase Database

Multi-Step Regression Learning for Compositional Distributional Semantics

Semantic Compositionality through Recursive Matrix-Vector Spaces

Improving Word Representations via Global Context and Multiple Word Prototypes

Improving neural networks by preventing co-adaptation of feature detectors

Compositional Matrix-Space Models for Sentiment Analysis

Parsing Natural Scenes and Natural Language with Recursive Neural Networks

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

Natural Language Processing (Almost) from Scratch

Composition in Distributional Models of Semantics

Word Representations: A Simple and General Method for Semi-Supervised Learning

Accurate Unlexicalized Parsing

The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions

Long Short-Term Memory

A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge.

Learning long-term dependencies with gradient descent is difficult

Finding Structure in Time

Learning representations by back-propagating errors

A Fast and Accurate Dependency Parser using Neural Networks

Dropout: a simple way to prevent neural networks from overfitting

Statistical Language Models Based on Neural Networks

The Measurement of Textual Coherence with Latent Semantic Analysis.

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names