Generating Sequences With Recurrent Neural Networks (2013-08-04T00:00:00.000000Z)

TL;DR

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time.

Abstract

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach is demonstrated for text (where the data are discrete) and online handwriting (where the data are real-valued). It is then extended to handwriting synthesis by allowing the network to condition its predictions on a text sequence. The resulting system is able to generate highly realistic cursive handwriting in a wide variety of styles.

Authors

Alex Graves

11 papers

TL;DR

Abstract

Authors

References38 items

Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets

Speech recognition with deep recurrent neural networks

Sequence Transduction with Recurrent Neural Networks

A fast and simple algorithm for training neural probabilistic language models

Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription

A Machine Learning Perspective on Predictive Coding with PAQ8

Practical Variational Inference for Neural Networks

A Machine Learning Perspective on Predictive Coding with PAQ

Generating Text with Recurrent Neural Networks

Factored conditional restricted Boltzmann Machines for modeling motion style

The Recurrent Temporal Restricted Boltzmann Machine

A Scalable Hierarchical Distributed Language Model

Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks

The Minimum Description Length Principle (Adaptive Computation and Machine Learning)

IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard

2005 Special Issue: Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Learning Precise Timing with LSTM Recurrent Networks

A First Look at Music Composition using LSTM Recurrent Neural Networks

Better Generative Models for Sequential Data Problems: Bidirectional Recurrent Mixture Density Networks

Long Short-Term Memory

An analysis of noise in recurrent neural networks: convergence and generalization

Learning long-term dependencies with gradient descent is difficult

Building a Large Annotated Corpus of English: The Penn Treebank

Data Compression Using Adaptive Coding and Partial String Matching

OF THE UNITED STATES

Statistical Language Models Based on Neural Networks

A Practical Guide to Training Restricted Boltzmann Machines

The Human Knowledge Compression Contest

Lecture 6.5 -rmsprop: Divide the gradient by a running average of its recent magnitude

SUBWORD LANGUAGE MODELING WITH NEURAL NETWORKS

Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies

Gradient-based learning algorithms for recurrent networks and their computational complexity

Mixture Density Networks

Neural Networks for Pattern Recognition

The tagged LOB Corpus : user's manual

and G

Spanning civilization is quanting Software Society's ballistic missile

B-45 BQ|B9

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names