A Temporal Variational Model for Story Generation (2021-09-14T00:00:00.000000Z)

TL;DR

Experiments with a latent vector planning approach based on a TD-VAE (Temporal Difference Variational Autoencoder) using the model for conditioning and reranking for text generation demonstrate strong performance in automatic cloze and swapping evaluations.

Abstract

Recent language models can generate interesting and grammatically correct text in story generation but often lack plot development and long-term coherence. This paper experiments with a latent vector planning approach based on a TD-VAE (Temporal Difference Variational Autoencoder), using the model for conditioning and reranking for text generation. The results demonstrate strong performance in automatic cloze and swapping evaluations. The human judgments show stories generated with TD-VAE reranking improve on a GPT-2 medium baseline and show comparable performance to a hierarchical LSTM reranking model. Conditioning on the latent vectors proves disappointing and deteriorates performance in human evaluation because it reduces the diversity of generation, and the models don't learn to progress the narrative. This highlights an important difference between technical task performance (e.g. cloze) and generating interesting stories.

Authors

Frank Keller

4 papers

David Wilmot

1 papers

TL;DR

Abstract

Authors

References64 items

OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics

The Cambridge Introduction to Narrative

Plan-CVAE: A Planning-based Conditional Variational Autoencoder for Story Generation

Content Planning for Neural Story Generation with Aristotelian Rescoring

UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation

Language Models are Few-Shot Learners

Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation

Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space

The Shmoop Corpus: A Dataset of Stories with Loosely Aligned Summaries

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Variational Temporal Abstraction

Do Massively Pretrained Language Models Make Better Storytellers?

Story Realization: Expanding Plot Events into Sentences

Semantics-aware BERT for Language Understanding

Encoder-Agnostic Adaptation for Conditional Language Generation

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Neural Text Generation with Unlikelihood Training

Learning to Write Stories with Thematic Consistency and Wording Novelty

Generating Diverse High-Fidelity Images with VQ-VAE-2

The Curious Case of Neural Text Degeneration

Plan, Write, and Revise: an Interactive System for Open-Domain Story Generation

Strategies for Structuring Story Generation

Plan-And-Write: Towards Better Automatic Storytelling

Recurrent World Models Facilitate Policy Evolution

A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation

Temporal Difference Variational Auto-Encoder

What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties

Hierarchical Neural Story Generation

Learning to Write with Cooperative Discriminators

Universal Sentence Encoder

AllenNLP: A Deep Semantic Natural Language Processing Platform

SentEval: An Evaluation Toolkit for Universal Sentence Representations

An efficient framework for learning sentence representations

Probabilistic Recurrent State-Space Models

Neural Discrete Representation Learning

Attention is All you Need

Event Representations for Automated Story Generation with Deep Neural Nets

Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference

The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task

Bag of Tricks for Efficient Text Classification

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

Possible Worlds

Writing Stories with Help from Recurrent Neural Networks

The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations

A large annotated corpus for learning natural language inference

Skip-Thought Vectors

Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books

A Recurrent Latent Variable Model for Sequential Data

Learning Stochastic Feedforward Neural Networks

Generating Sequences With Recurrent Neural Networks

Learning Latent Personas of Film Characters

Story Generation with Crowdsourced Plot Graphs

New Alignment Methods for Discriminative Book Summarization

All the World's a Stage: Learning Character Models from Film

Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning

Narrativity and Eventfulness

Long Short-Term Memory

The Structure of the Artistic Text

AUTO-ENCODING VARIATIONAL BAYES

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Language Models are Unsupervised Multitask Learners

Improving Language Understanding by Generative Pre-Training

Atomic: An atlas of machine commonsense for if-then reasoning

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names