Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speech

Published in

arXiv.org(2021)

External Links:

Generate Graph

TL;DR

Experimental results show that the proposed cross-speaker emotion transfer method outperforms the multi-reference based baseline in terms of timbre similarity, stability and emotion perceive evaluations.

Abstract

In expressive speech synthesis, there are high requirements for emotion interpretation. However, it is time-consuming to acquire emotional audio corpus for arbitrary speakers due to their deduction ability. In response to this problem, this paper proposes a cross-speaker emotion transfer method that can realize the transfer of emotions from source speaker to target speaker. A set of emotion tokens is firstly defined to represent various categories of emotions. They are trained to be highly correlated with corresponding emotions for controllable synthesis by cross-entropy loss and semi-supervised training strategy. Meanwhile, to eliminate the down-gradation to the timbre similarity from cross-speaker emotion transfer, speaker condition layer normalization is implemented to model speaker characteristics. Experimental results show that the proposed method outperforms the multi-reference based baseline in terms of timbre similarity, stability and emotion perceive evaluations.

Authors

Pengfei Wu

1 papers

Junjie Pan

1 papers

Chenchang Xu

1 papers

References22 items

AdaSpeech: Adaptive Text to Speech for Custom Voice

Parallel Tacotron: Non-Autoregressive and Controllable TTS

Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistency

Semi-Supervised Generative Modeling for Controllable Speech Synthesis

End-to-End Emotional Speech Synthesis Using Style Tokens and Semi-Supervised Training

Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speech

Published in

arXiv.org(2021)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Pengfei Wu

1 papers

Junjie Pan

1 papers

Chenchang Xu

1 papers

References22 items

AdaSpeech: Adaptive Text to Speech for Custom Voice

Parallel Tacotron: Non-Autoregressive and Controllable TTS

Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistency

Semi-Supervised Generative Modeling for Controllable Speech Synthesis

End-to-End Emotional Speech Synthesis Using Style Tokens and Semi-Supervised Training

Junhui Zhang

1 papers

Multi-reference Tacotron by Intercross Training for Style Disentangling, Transfer and Control in Speech Synthesis

Learning Latent Representations for Style Control and Transfer in End-to-end Speech Synthesis

Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder

Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

Efficient Neural Audio Synthesis

Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

Emotional statistical parametric speech synthesis using LSTM-RNNs

Emotional End-to-End Neural Speech Synthesizer

Tacotron: Towards End-to-End Speech Synthesis

Deep Voice: Real-time Neural Text-to-Speech

Char2Wav: End-to-End Speech Synthesis

SampleRNN: An Unconditional End-to-End Neural Audio Generation Model

WaveNet: A Generative Model for Raw Audio

Controllable cross-speaker emotion transfer for end-to-end speech synthesis

TTS synthesis with bidirectional LSTM based recurrent neural networks

Visualizing Data using t-SNE

Field of Study

EngineeringComputer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

arXiv.org

Type

URL

https://arxiv.org

Alternate Names

ArXiv