Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speech (2021-03-06T00:00:00.000000Z)

TL;DR

This work investigates different speaker representations and proposed to integrate pretrained and learnable speaker representations to find the embedding pretrained by voice conversion achieves the best performance.

Abstract

The few-shot multi-speaker multi-style voice cloning task is to synthesize utterances with voice and speaking style similar to a reference speaker given only a few reference samples. In this work, we investigate different speaker representations and proposed to integrate pretrained and learnable speaker representations. Among different types of embeddings, the embedding pretrained by voice conversion achieves the best performance. The FastSpeech 2 model combined with both pretrained and learnable speaker representations shows great generalization ability on few-shot speakers and achieved 2nd place in the one-shot track of the ICASSP 2021 M2VoC challenge.

Authors

Hung-yi Lee

13 papers

Chien-yu Huang

2 papers

C. Chien

1 papers

Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speech

TL;DR

Abstract

Authors

References21 items

The Multi-Speaker Multi-Style Voice Cloning Challenge 2021

Controllable Emotion Transfer For End-to-End Speech Synthesis

FastSpeech 2: Fast and High-Quality End-to-End Text to Speech

From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint

Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings

Multi-Speaker End-to-End Speech Synthesis

End-to-End Emotional Speech Synthesis Using Style Tokens and Semi-Supervised Training

One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization

Hierarchical Generative Modeling for Controllable Speech Synthesis

Sample Efficient Adaptive Text-to-Speech

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

X-Vectors: Robust DNN Embeddings for Speaker Recognition

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

Neural Voice Cloning with a Few Samples

Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

Generalized End-to-End Loss for Speaker Verification

Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi

WaveNet: A Generative Model for Raw Audio

The Ninth International Conference on Machine Learning

Aäron van den Oord, Oriol Vinyals, and Nando de Freitas

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names