OverFlow: Putting flows on top of neural transducers for better TTS (2022-11-13T00:00:00.000000Z)

TL;DR

Experiments show that a system based on the proposed neural HMM TTS with normalising flows for describing the highly non-Gaussian distribution of speech acoustics needs fewer updates than comparable methods to produce accurate pronunciations and a subjective speech quality close to natural speech.

Abstract

Neural HMMs are a type of neural transducer recently proposed for sequence-to-sequence modelling in text-to-speech. They combine the best features of classic statistical speech synthesis and modern neural TTS, requiring less data and fewer training updates, and are less prone to gibberish output caused by neural attention failures. In this paper, we combine neural HMM TTS with normalising flows for describing the highly non-Gaussian distribution of speech acoustics. The result is a powerful, fully probabilistic model of durations and acoustics that can be trained using exact maximum likelihood. Experiments show that a system based on our proposal needs fewer updates than comparable methods to produce accurate pronunciations and a subjective speech quality close to natural speech. Please see https://shivammehta25.github.io/OverFlow/ for audio examples and code.

Authors

G. Henter

8 papers

J. Beskow

3 papers

Shivam Mehta

2 papers

TL;DR

Abstract

Authors

References44 items

Robust Speech Recognition via Large-Scale Weak Supervision

Prosody-Controllable Spontaneous TTS with Neural HMMS

Neural HMMS Are All You Need (For High-Quality Attention-Free TTS)

Confidence Intervals for ASR-Based TTS Evaluation

One TTS Alignment to Rule Them All

Variational Diffusion Models

Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech

EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecture

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Robust Classification Using Hidden Markov Models and Mixtures of Normalizing Flows

Fastpitch: Parallel Text-to-Speech with Pitch Prediction

Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search

Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis

Normalizing Flows: An Introduction and Review of Current Methods

Flow-TTS: A Non-Autoregressive Network for Text to Speech Based on Flow

Normalizing Flows for Probabilistic Modeling and Inference

Initial investigation of encoder-decoder end-to-end TTS using marginalization of monotonic hard alignments

Where do the improvements come from in sequence-to-sequence neural TTS?

Spontaneous Conversational Speech Synthesis from Found Data

Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS

Waveglow: A Flow-based Generative Network for Speech Synthesis

Neural Speech Synthesis with Transformer Network

Glow: Generative Flow with Invertible 1x1 Convolutions

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

Attention is All you Need

Tacotron: Towards End-to-End Speech Synthesis

Non-parametric duration modelling for speech synthesis with a joint model of acoustics and duration

Unsupervised Neural Hidden Markov Models

Online Segment to Segment Neural Transduction

Median-based generation of synthetic speech durations using a non-parametric approach

Autoregressive Models for Statistical Parametric Speech Synthesis

A tutorial on hidden Markov models and selected applications in speech recognition

Speech-T: Transducer for Text to Speech and Beyond

RAD-TTS: Parallel Flow-Based TTS with Robust Alignment Learning and Diverse Synthesis

Loudness normalisation and permitted maximum level of audio signals

Method for the subjective assessment of intermedi-ate quality levels of coding systems

A Simple Sequentially Rejective Multiple Test Procedure

IEEE Recommended Practice for Speech Quality Measurements

Edinburgh Research Explorer Measuring the Perceptual Effects of Modelling Assumptions in Speech Synthesis Using Stimuli Constructed from Repeated Natural Speech

METHODS FOR SUBJECTIVE DETERMINATION OF TRANSMISSION QUALITY Summary

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names