Stochastic Pitch Prediction Improves the Diversity and Naturalness of Speech in Glow-TTS (2023-05-28T00:00:00.000000Z)

TL;DR

This paper proposes to improve the diversity of utterances by explicitly learning the distribution of fundamental frequency sequences (pitch contours) of each speaker during training using a stochastic flow-based pitch predictor, then conditioning the model on generated pitch contours during inference.

Abstract

Flow-based generative models are widely used in text-to-speech (TTS) systems to learn the distribution of audio features (e.g., Mel-spectrograms) given the input tokens and to sample from this distribution to generate diverse utterances. However, in the zero-shot multi-speaker TTS scenario, the generated utterances lack diversity and naturalness. In this paper, we propose to improve the diversity of utterances by explicitly learning the distribution of fundamental frequency sequences (pitch contours) of each speaker during training using a stochastic flow-based pitch predictor, then conditioning the model on generated pitch contours during inference. The experimental results demonstrate that the proposed method yields a significant improvement in the naturalness and diversity of speech generated by a Glow-TTS model that uses explicit stochastic pitch prediction, over a Glow-TTS baseline and an improved Glow-TTS model that uses a stochastic duration predictor.

Authors

E. Vincent

6 papers

Sewade Ogun

1 papers

Vincent Colotte

1 papers

TL;DR

Abstract

Authors

References30 items

SNAC: Speaker-Normalized Affine Coupling Layer in Flow-Based Architecture for Zero-Shot Multi-Speaker Text-to-Speech

Can We Use Common Voice to Train a Multi-Speaker TTS System?

Varianceflow: High-Quality and Controllable Text-to-Speech using Variance Information via Normalizing Flow

Revisiting Over-Smoothness in Text to Speech

Normalization Driven Zero-Shot Multi-Speaker Speech Synthesis

Improving transfer of expressivity for end-to-end multispeaker text-to-speech synthesis

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation

SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model

Expressive Text-to-Speech Using Style Tag

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Fastpitch: Parallel Text-to-Speech with Pitch Prediction

FastSpeech 2: Fast and High-Quality End-to-End Text to Speech

Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search

Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis

Flow-TTS: A Non-Autoregressive Network for Text to Speech Based on Flow

Voxceleb: Large-scale speaker verification in the wild

VFlow: More Expressive Generative Flows with Variational Data Augmentation

Common Voice: A Massively-Multilingual Speech Corpus

Mellotron: Multispeaker Expressive Voice Synthesis by Conditioning on Rhythm, Pitch and Global Style Tokens

NeMo: a toolkit for building AI applications using Neural Modules

On the Variance of the Adaptive Learning Rate and Beyond

Neural Spline Flows

LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech

Representation Mixing for TTS Synthesis

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

SUPERSEDED - CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit

PYIN: A fundamental frequency estimator using probabilistic threshold distributions

HiFi++: a Unified Framework for Neural Vocoding, Bandwidth Extension and Speech Enhancement

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names