YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone
Published in
International Conference on Machine Learning(2021)External Links:
TL
TL;DR
The YourTTS model builds upon the VITS model and adds several novel modifications for zero-shot multi-speaker and multilingual training, achieving state-of-the-art (SOTA) results in zero- shot multi- Speaker TTS and results comparable to SOTA in zero -shot voice conversion on the VCTK dataset.
Abstract
YourTTS brings the power of a multilingual approach to the task of zero-shot multi-speaker TTS. Our method builds upon the VITS model and adds several novel modifications for zero-shot multi-speaker and multilingual training. We achieved state-of-the-art (SOTA) results in zero-shot multi-speaker TTS and results comparable to SOTA in zero-shot voice conversion on the VCTK dataset. Additionally, our approach achieves promising results in a target language with a single-speaker dataset, opening possibilities for zero-shot multi-speaker TTS and zero-shot voice conversion systems in low-resource languages. Finally, it is possible to fine-tune the YourTTS model with less than 1 minute of speech and achieve state-of-the-art results in voice similarity and with reasonable quality. This is important to allow synthesis for speakers with a very different voice or recording characteristics from those seen during training.
References48 items
1
Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations
2
Normalization Driven Zero-Shot Multi-Speaker Speech Synthesis
3
Cross-Lingual Speaker Adaptation Using Domain Adaptation and Speaker Consistency Loss for Text-To-Speech Synthesis
4
A Survey on Neural Speech Synthesis
5