Matcha-TTS: A Fast TTS Architecture with Conditional Flow Matching (2023-09-06T00:00:00.000000Z)

TL;DR

Matcha-TTS, a new encoder-decoder architecture for speedy TTS acoustic modelling, trained using optimal-transport conditional flow matching (OT-CFM), is introduced, an ODE-based decoder capable of high output quality in fewer synthesis steps than models trained using score matching.

Abstract

We introduce Matcha-TTS, a new encoder-decoder architecture for speedy TTS acoustic modelling, trained using optimal-transport conditional flow matching (OT-CFM). This yields an ODE-based decoder capable of high output quality in fewer synthesis steps than models trained using score matching. Careful design choices additionally ensure each synthesis step is fast to run. The method is probabilistic, non-autoregressive, and learns to speak from scratch without external alignments. Compared to strong pre-trained baseline models, the Matcha-TTS system has the smallest memory footprint, rivals the speed of the fastest model on long utterances, and attains the highest mean opinion score in a listening test.

Authors

G. Henter

8 papers

J. Beskow

3 papers

Shivam Mehta

2 papers

TL;DR

Abstract

Authors

References43 items

Stuck in the MOS pit: A critical analysis of MOS test methodology in TTS evaluation

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale

Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis

Why We Should Report the Details in Subjective Evaluation of TTS More Rigorously

Better speech synthesis through scaling

CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model

Robust Speech Recognition via Large-Scale Weak Supervision

Grad-StyleSpeech: Any-Speaker Adaptive Text-to-Speech Synthesis with Diffusion Models

Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models

Emodiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance

OverFlow: Putting flows on top of neural transducers for better TTS

Flow Matching for Generative Modeling

Building Normalizing Flows with Stochastic Interpolants

Fast Grad-TTS: Towards Efficient Diffusion-Based Speech Generation on CPU

Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech

BigVGAN: A Universal Neural Vocoder with Large-Scale Training

High-Resolution Image Synthesis with Latent Diffusion Models

Phonemizer: Text to Phones Transcription for Multiple Languages in Python

Neural HMMS Are All You Need (For High-Quality Attention-Free TTS)

LinearSpeech: Parallel Text-to-Speech with Linear Complexity

Confidence Intervals for ASR-Based TTS Evaluation

Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation

WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models

Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech

Diffusion Models Beat GANs on Image Synthesis

RoFormer: Enhanced Transformer with Rotary Position Embedding

Diff-TTS: A Denoising Diffusion Model for Text-to-Speech

Score-Based Generative Modeling through Stochastic Differential Equations

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

DiffWave: A Versatile Diffusion Model for Audio Synthesis

WaveGrad: Estimating Gradients for Waveform Generation

FastSpeech 2: Fast and High-Quality End-to-End Text to Speech

Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search

Where do the improvements come from in sequence-to-sequence neural TTS?

Spontaneous Conversational Speech Synthesis from Found Data

Generative Modeling by Estimating Gradients of the Data Distribution

Waveglow: A Flow-based Generative Network for Speech Synthesis

Neural Ordinary Differential Equations

Self-Attention with Relative Position Representations

The Blizzard Challenge 2013 - Indian Language Task

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names