WaveNet: A Generative Model for Raw Audio (2016-09-12T00:00:00.000000Z)

TL;DR

WaveNet, a deep neural network for generating raw audio waveforms, is introduced; it is shown that it can be efficiently trained on data with tens of thousands of samples per second of audio, and can be employed as a discriminative model, returning promising results for phoneme recognition.

Abstract

This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.

Authors

O. Vinyals

41 papers

K. Simonyan

25 papers

Nal Kalchbrenner

11 papers

TL;DR

Abstract

Authors

References64 items

SUPERSEDED - CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit

Recent Advances in Google Real-Time HMM-Driven Unit Selection Synthesizer

WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications

Fast, Compact, and High Quality LSTM-RNN Based Statistical Parametric Speech Synthesizers for Mobile Devices

Conditional Image Generation with PixelCNN Decoders

Postfilters to Modify the Modulation Spectrum for Statistical Parametric Speech Synthesis

Directly modeling voiced and unvoiced components in speech waveforms by neural networks

A deep auto-encoder based low-dimensional feature extraction from FFT spectral envelopes for statistical parametric speech synthesis

Exploring the Limits of Language Modeling

Pixel Recurrent Neural Networks

Deep Residual Learning for Image Recognition

Multi-Scale Context Aggregation by Dilated Convolutions

Generative Image Modeling Using Spatial LSTMs

Vocaine the vocoder and applications in speech synthesis

Directly modeling speech waveforms by neural networks for statistical parametric speech synthesis

Speech acoustic modeling from raw multichannel waveforms

Modelling acoustic feature dependencies with artificial neural networks: Trajectory-RNADE

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

A Deep Learning Approach to Data-driven Parameterizations for Statistical Parametric Speech Synthesis

Integration of Spectral Feature Extraction and Modeling for HMM-Based Speech Synthesis

Statistical parametric speech synthesis using deep neural networks

Estimating phoneme class conditional probabilities from raw speech signal using convolutional neural networks

Statistical parametric speech synthesis with joint estimation of acoustic and excitation model parameters

Rectified Linear Units Improve Restricted Boltzmann Machines

Input-agreement: a new mechanism for collecting data using human computation games

Statistical approach to vocal tract transfer function estimation based on factor analyzed trajectory HMM

A Speech Parameter Generation Algorithm Considering Global Variance for HMM-Based Speech Synthesis

Statistical Parametric Speech Synthesis

Reformulating the HMM as a trajectory model by imposing explicit relationships between static and dynamic feature vector sequences

Nonlinear filter design: methodologies and challenges

Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds

Long Short-Term Memory

Text-to-speech conversion with neural networks: a recurrent TDNN approach

Unit selection in a concatenative speech synthesis system using a large speech database

Speech synthesis using artificial neural networks trained on cepstral coefficients

DARPA TIMIT:: acoustic-phonetic continuous speech corpus CD-ROM, NIST speech disc 1-1.1

ATR μ-talk speech synthesis system

Wavelets: Time-Frequency Methods and Phase Space

Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones

Mixture autoregressive hidden Markov models for speech signals

Linear predictive hidden Markov models and the speech signal

Line spectrum representation of linear predictor coefficients of speech signals

Learning the speech front-end with raw waveform CLDNNs

TTS synthesis with bidirectional LSTM based recurrent neural networks

Acoustic modeling with deep neural networks using raw time signal for LVCSR

Speech synthesis as a statistical machine learning problem. http://www.sp

Speech analysis with multi-kernel linear prediction

Minimum generation error training with direct log spectral distortion on LSPs for HMM-based speech synthesis

An example of context-dependent label format for HMM-based speech synthesis in English

Reformulating HMM as a trajectory model by imposing explicit relationships between static and dynamic features

Simultaneous modeling of phonetic and prosodic parameters,and characteristic conversion for HMM-based text-to-speech systems

Aperiodicity extraction and control using mixed mode excitation and group delay manipulation for a high quality speech analysis, modification and synthesis system STRAIGHT

Mixture Density Networks

Fundamentals of speech recognition

A real-time algorithm for signal analysis with the help of the wavelet transform

An Implementation of the “algorithme à trous” to Compute the Wavelet Transform

Unbiased estimation of log spectrum

Kameoka et al., 2010) within a windowed frame. However, the relationship between successive audio samples can be highly non-linear

Pulse Code Modulation (PCM) of voice frequencies

A statistical method for estimation of speech spectral density and formant frequencies

These generative models are typically realized as a linear time-invariant filter (Itakura & Saito

Acoustic Theory Of Speech Production

The vowel, its nature and structure

Remaking speech

Field of Study

Venue Information

Name

Type

URL

Alternate Names