ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (2017-11-15T00:00:00.000000Z)

TL;DR

This work uses ParaNMT-50M, a dataset of more than 50 million English-English sentential paraphrase pairs, to train paraphrastic sentence embeddings that outperform all supervised systems on every SemEval semantic textual similarity competition, in addition to showing how it can be used for paraphrase generation.

Abstract

We describe ParaNMT-50M, a dataset of more than 50 million English-English sentential paraphrase pairs. We generated the pairs automatically by using neural machine translation to translate the non-English side of a large parallel corpus, following Wieting et al. (2017). Our hope is that ParaNMT-50M can be a valuable resource for paraphrase generation and can provide a rich source of semantic knowledge to improve downstream natural language understanding tasks. To show its utility, we use ParaNMT-50M to train paraphrastic sentence embeddings that outperform all supervised systems on every SemEval semantic textual similarity competition, in addition to showing how it can be used for paraphrase generation.

Authors

J. Wieting

2 papers

Kevin Gimpel

1 papers

TL;DR

Abstract

Authors

References70 items

Adversarial Example Generation with Syntactically Controlled Paraphrase Networks

Automatic differentiation in PyTorch

Learned in Translation: Contextualized Word Vectors

A Continuously Growing Dataset of Sentential Paraphrases

SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation

Building a Non-Trivial Paraphrase Corpus Using Multiple Machine Translation Systems

Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext

Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings

A Simple but Tough-to-Beat Baseline for Sentence Embeddings

Understanding Task Design Trade-offs in Crowdsourced Paraphrase Collection

A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference

Paraphrasing Revisited with Neural Machine Translation

Learning to Generate Reviews and Discovering Sentiment

Nematus: a Toolkit for Neural Machine Translation

Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features

Learning Generic Sentence Representations Using Convolutional Neural Networks

CzEng 1.6: Enlarged Czech-English Parallel Corpus with Processing Tools Dockered

Charagram: Embedding Words and Sentences via Character n-grams

Theano: A Python framework for fast computation of mathematical expressions

Counter-fitting Word Vectors to Linguistic Constraints

Learning Distributed Representations of Sentences from Unlabelled Data

Towards Universal Paraphrastic Sentence Embeddings

A large annotated corpus for learning natural language inference

Jointly optimizing word representations for lexical and sentential tasks with the C-PHRASE model

Skip-Thought Vectors

From Paraphrase Database to Compositional Paraphrase Model and Back

SemEval-2015 Task 1: Paraphrase and Semantic Similarity in Twitter (PIT)

SemEval-2015 Task 2: Semantic Textual Similarity, English, Spanish and Pilot on Interpretability

Self-Adaptive Hierarchical Sentence Model

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Adam: A Method for Stochastic Optimization

Embedding Word Similarity with Neural Machine Translation

Extracting Lexically Divergent Paraphrases from Twitter

Not All Neural Embeddings are Born Equal

GloVe: Global Vectors for Word Representation

Sequence to Sequence Learning with Neural Networks

Neural Machine Translation by Jointly Learning to Align and Translate

SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation

Illinois-LH: A Denotational and Distributional Approach to Semantics

SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment

SemEval-2014 Task 10: Multilingual Semantic Textual Similarity

The Stanford CoreNLP Natural Language Processing Toolkit

Meteor Universal: Language Specific Translation Evaluation for Any Target Language

Distributed Representations of Sentences and Documents

The Multilingual Paraphrase Database

Learning deep structured semantic models for web search using clickthrough data

Distributed Representations of Words and Phrases and their Compositionality

Discriminative Improvements to Distributional Sentence Similarity

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

*SEM 2013 shared task: Semantic Textual Similarity

PPDB: The Paraphrase Database

SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity

Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection

Simple English Wikipedia: A New Text Simplification Task

Paraphrasing with Bilingual Parallel Corpora

Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales

Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources

Mining and summarizing customer reviews

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts

Monolingual Machine Translation for Paraphrase Generation

Bleu: a Method for Automatic Evaluation of Machine Translation

Discovery of inference rules for question-answering

Extracting Paraphrases from a Parallel Corpus

Long Short-Term Memory

HCTI at SemEval-2017 Task 1: Use convolutional neural network to evaluate Semantic Textual Similarity

SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation

Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Automatically Constructing a Corpus of Sentential Paraphrases

International Conference on Learning Represen-tations

Field of Study

Venue Information

Name

Type

URL

Alternate Names