One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization

Published in

Interspeech(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

This paper proposed a novel one-shot VC approach which is able to perform VC by only an example utterance from source and target speaker respectively, and the source andtarget speaker do not even need to be seen during training.

Abstract

Recently, voice conversion (VC) without parallel data has been successfully adapted to multi-target scenario in which a single model is trained to convert the input voice to many different speakers. However, such model suffers from the limitation that it can only convert the voice to the speakers in the training data, which narrows down the applicable scenario of VC. In this paper, we proposed a novel one-shot VC approach which is able to perform VC by only an example utterance from source and target speaker respectively, and the source and target speaker do not even need to be seen during training. This is achieved by disentangling speaker and content representations with instance normalization (IN). Objective and subjective evaluation shows that our model is able to generate the voice similar to target speaker. In addition to the performance measurement, we also demonstrate that this model is able to learn meaningful speaker representations without any supervision.

Authors

Hung-yi Lee

13 papers

Ju-Chieh Chou

2 papers

Cheng-chieh Yeh

1 papers

References36 items

Generative Adversarial Networks

Disentangling Correlated Speaker and Noise for Speech Synthesis via Data Augmentation and Adversarial Factorization

GANSynth: Adversarial Neural Audio Synthesis

Learning Speaker Representations with Mutual Information

Generative Adversarial Networks for Unpaired Voice Transformation on Impaired Speech

One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization

Published in

Interspeech(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Hung-yi Lee

13 papers

Ju-Chieh Chou

2 papers

Cheng-chieh Yeh

1 papers

References36 items

Generative Adversarial Networks

Disentangling Correlated Speaker and Noise for Speech Synthesis via Data Augmentation and Adversarial Factorization

GANSynth: Adversarial Neural Audio Synthesis

Learning Speaker Representations with Mutual Information

Generative Adversarial Networks for Unpaired Voice Transformation on Impaired Speech

Synthetic-to-Natural Speech Waveform Conversion Using Cycle-Consistent Adversarial Networks

Automatic Speech Pronunciation Correction with Dynamic Frequency Warping-Based Spectral Conversion

ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder

Rhythm-Flexible Voice Conversion Without Parallel Data Using Cycle-GAN Over Phoneme Posteriorgram Sequences

StarGAN-VC: non-parallel many-to-many Voice Conversion Using Star Generative Adversarial Networks

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations

Machine Speech Chain with One-shot Speaker Adaptation

Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks

Non-parallel voice conversion using i-vector PLDA: towards unifying speaker verification and transformation

Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities

Voice Conversion from Unaligned Corpora Using Variational Autoencoding Wasserstein Generative Adversarial Networks

Tacotron: Towards End-to-End Speech Synthesis

Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization

Unsupervised Image-to-Image Translation Networks

Generative adversarial network-based postfilter for statistical parametric speech synthesis

SUPERSEDED - CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit

Personalized, Cross-Lingual TTS Using Phonetic Posteriorgrams

A KL Divergence and DNN-Based Approach to Voice Conversion without Parallel Training Sentences

Improving the intelligibility of dysarthric speech towards enhancing the effectiveness of speech therapy

Instance Normalization: The Missing Ingredient for Fast Stylization

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Voice conversion using deep Bidirectional Long Short-Term Memory based Recurrent Neural Networks

Voice Conversion Using Deep Neural Networks With Layer-Wise Generative Training

Voice Conversion Based on Speaker-Dependent Restricted Boltzmann Machines

Voice conversion using deep neural networks with speaker-independent pre-training

Auto-Encoding Variational Bayes

A Speech Parameter Generation Algorithm Considering Global Variance for HMM-Based Speech Synthesis

High-order sequence modeling using speaker-dependent recurrent temporal restricted boltzmann machines for voice conversion

Field of Study

Computer ScienceEngineeringMathematics

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

Interspeech

Type

conference

URL

https://www.isca-speech.org/iscaweb/index.php/conferences/interspeech

Alternate Names

Conf Int Speech Commun Assoc
INTERSPEECH
Conference of the International Speech Communication Association