Neural Paraphrase Generation with Stacked Residual LSTM Networks

Published in

International Conference on Computational Lingu...(2016)

External Links:

Generate Graph

TL;DR

This work is the first to explore deep learning models for paraphrase generation with a stacked residual LSTM network, where it adds residual connections between L STM layers for efficient training of deep LSTMs.

Abstract

In this paper, we propose a novel neural approach for paraphrase generation. Conventional paraphrase generation methods either leverage hand-written rules and thesauri-based alignments, or use statistical machine learning principles. To the best of our knowledge, this work is the first to explore deep learning models for paraphrase generation. Our primary contribution is a stacked residual LSTM network, where we add residual connections between LSTM layers. This allows for efficient training of deep LSTMs. We evaluate our model and other state-of-the-art deep learning models on three different datasets: PPDB, WikiAnswers, and MSCOCO. Evaluation results demonstrate that our model outperforms sequence to sequence, attention-based, and bi-directional LSTM models on BLEU, METEOR, TER, and an embedding-based sentence similarity metric.

Authors

Sadid A. Hasan

2 papers

Aaditya (Adi) Prakash

2 papers

Kathy Lee

1 papers

References53 items

Word2Vec

Densely Connected Convolutional Networks

Full Resolution Image Compression with Recurrent Neural Networks

Generating Natural Language Inference Chains

Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

Neural Paraphrase Generation with Stacked Residual LSTM Networks

Published in

International Conference on Computational Lingu...(2016)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Sadid A. Hasan

2 papers

Aaditya (Adi) Prakash

2 papers

Kathy Lee

1 papers

References53 items

Word2Vec

Densely Connected Convolutional Networks

Full Resolution Image Compression with Recurrent Neural Networks

Generating Natural Language Inference Chains

Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

Vivek Datla

1 papers

Ashequl Qadir

1 papers

Oladimeji Farri

1 papers

Theano: A Python framework for fast computation of mathematical expressions

Deep Residual Learning for Image Recognition

PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification

Skip-Thought Vectors

From Paraphrase Database to Compositional Paraphrase Model and Back

End-To-End Memory Networks

Grammar as a Foreign Language

Show and tell: A neural image caption generator

Neural Turing Machines

Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition

Sequence to Sequence Learning with Neural Networks

Very Deep Convolutional Networks for Large-Scale Image Recognition

Neural Machine Translation by Jointly Learning to Align and Translate

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Microsoft COCO: Common Objects in Context

How to Construct Deep Recurrent Neural Networks

Hybrid speech recognition with Deep Bidirectional LSTM

Generating Sequences With Recurrent Neural Networks

Paraphrase-Driven Learning for Open Question Answering

Learning Hierarchical Features for Scene Labeling

A Comparison of Greedy and Optimal Assessment of Natural Language Student Input Using Word-to-Word Similarity Metrics

Re-examining Machine Translation Metrics for Paraphrase Identification

Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection

Generating Text with Recurrent Neural Networks

Collecting Highly Parallel Data for Paraphrase Evaluation

Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability

PEM: A Paraphrase Evaluation Metric Exploiting Parallel Texts

Generating Phrasal and Sentential Paraphrases: A Survey of Data-Driven Methods

Leveraging Multiple MT Engines for Paraphrase Generation

Paraphrase Generation as Monolingual Translation: Data and Evaluation

Application-driven Statistical Paraphrase Generation

Combining Multiple Resources to Improve SMT-based Paraphrasing Model

UNT: SubFinder: Combining Knowledge Sources for Automatic Lexical Substitution

METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments

Paraphrasing with Bilingual Parallel Corpora

Monolingual Machine Translation for Paraphrase Generation

Generation of Single-sentence Paraphrases from Predicate/Argument Structure using Lexico-grammatical Resources

Bleu: a Method for Automatic Evaluation of Machine Translation

Long Short-Term Memory

Learning long-term dependencies with gradient descent is difficult

Convolutional Neural Network for Paraphrase Identification

LSTM Implementation Explained: apaszke.github.io/lstm-explained

Dropout: a simple way to prevent neural networks from overfitting

LSTM Neural Networks for Language Modeling

A Study of Translation Edit Rate with Targeted Human Annotation

A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION

Microsoft Research Paraphrase Corpus

Paraphrasing Questions Using Given and new information

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

International Conference on Computational Linguistics

Type

conference

URL

https://www.aclweb.org/anthology/venues/coling/

Alternate Names

Int Conf Comput Linguistics
COLING