Generating Wikipedia by Summarizing Long Sequences
Published in
International Conference on Learning Representa...(2018)External Links:
TL
TL;DR
It is shown that generating English Wikipedia articles can be approached as a multi- document summarization of source documents and a neural abstractive model is introduced, which can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles.
Abstract
We show that generating English Wikipedia articles can be approached as a multi- document summarization of source documents. We use extractive summarization to coarsely identify salient information and a neural abstractive model to generate the article. For the abstractive model, we introduce a decoder-only architecture that can scalably attend to very long sequences, much longer than typical encoder- decoder architectures used in sequence transduction. We show that this model can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles. When given reference documents, we show it can extract relevant factual information as reflected in perplexity, ROUGE scores and human evaluations.
References22 items
1
Attention is All you Need
2
A Deep Reinforced Model for Abstractive Summarization
3
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
4
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
5