WaveGrad: Estimating Gradients for Waveform Generation (2020-09-02T00:00:00.000000Z)

TL;DR

WaveGrad offers a natural way to trade inference speed for sample quality by adjusting the number of refinement steps, and bridges the gap between non-autoregressive and autoregressive models in terms of audio quality.

Abstract

This paper introduces WaveGrad, a conditional model for waveform generation which estimates gradients of the data density. The model is built on prior work on score matching and diffusion probabilistic models. It starts from a Gaussian white noise signal and iteratively refines the signal via a gradient-based sampler conditioned on the mel-spectrogram. WaveGrad offers a natural way to trade inference speed for sample quality by adjusting the number of refinement steps, and bridges the gap between non-autoregressive and autoregressive models in terms of audio quality. We find that it can generate high fidelity audio samples using as few as six iterations. Experiments reveal WaveGrad to generate high fidelity audio, outperforming adversarial non-autoregressive baselines and matching a strong likelihood-based autoregressive baseline using fewer sequential operations. Audio samples are available at this https URL.

Authors

Mohammad Norouzi

14 papers

Ron J. Weiss

14 papers

H. Zen

6 papers

TL;DR

Abstract

Authors

References66 items

DiffWave: A Versatile Diffusion Model for Audio Synthesis

Learning Gradient Fields for Shape Generation

HooliGAN: Robust, High Quality Neural Vocoding

A Spectral Energy Distance for Parallel Speech Synthesis

VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Network

Denoising Diffusion Probabilistic Models

Improved Techniques for Training Score-Based Generative Models

WaveNODE: A Continuous Normalizing Flow for Speech Synthesis

Multi-Band Melgan: Faster Waveform Generation For High-Quality Text-To-Speech

WaveFFJORD: FFJORD-Based Vocoder for Statistical Parametric Speech Synthesis

Non-Autoregressive Machine Translation with Latent Alignments

Knowledge-and-Data-Driven Amplitude Spectrum Prediction for Hierarchical Neural Vocoders

Imputer: Sequence Modelling via Imputation and Dynamic Programming

Insertion-Deletion Transformer

DDSP: Differentiable Digital Signal Processing

An Empirical Study of Generation Order for Machine Translation

Parallel Wavegan: A Fast Waveform Generation Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram

Location-Relative Attention Mechanisms for Robust Long-Form Speech Synthesis

MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis

Big Bidirectional Insertion Representations for Documents

High Fidelity Speech Synthesis with Adversarial Networks

Generative Modeling by Estimating Gradients of the Data Distribution

KERMIT: Generative Insertion-Based Modeling for Sequences

MelNet: A Generative Model for Audio in the Frequency Domain

GlotNet—A Raw Waveform Model for the Glottal Excitation in Statistical Parametric Speech Synthesis

Levenshtein Transformer

Non-Autoregressive Neural Text-to-Speech

Sliced Score Matching: A Scalable Approach to Density and Score Estimation

Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis

Direct speech-to-speech translation with a sequence-to-sequence model

Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation

Mask-Predict: Parallel Decoding of Conditional Masked Language Models

Semantic Image Synthesis With Spatially-Adaptive Normalization

Insertion Transformer: Flexible Sequence Generation via Insertion Operations

GANSynth: Adversarial Neural Audio Synthesis

ExcitNet Vocoder: A Neural Excitation Model for Parametric Speech Synthesis Systems

FloWaveNet : A Generative Flow for Raw Audio

Waveglow: A Flow-based Generative Network for Speech Synthesis

LPCNET: Improving Neural Speech Synthesis through Linear Prediction

Optimal Completion Distillation for Sequence Learning

ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech

Feature-wise transformations

Deep Energy Estimator Networks

Efficient Neural Audio Synthesis

Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement

Adversarial Audio Synthesis

Synthesizing Audio with Generative Adversarial Networks

Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

Parallel WaveNet: Fast High-Fidelity Speech Synthesis

Attention is All you Need

Tacotron: Towards End-to-End Speech Synthesis

Char2Wav: End-to-End Speech Synthesis

PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications

SampleRNN: An Unconditional End-to-End Neural Audio Generation Model

WaveNet: A Generative Model for Raw Audio

Deep Residual Learning for Image Recognition

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

A Connection Between Score Matching and Denoising Autoencoders

Reducing F0 Frame Error of F0 tracking algorithms under noisy conditions with an unvoiced/voiced classification frontend

Estimation of Non-Normalized Statistical Models by Score Matching

Mel-cepstral distance measure for objective speech quality assessment

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Multilingual KERMIT: Its Not Easy Being Generative

The LJ Speech Dataset

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names