Cyclegan-VC2: Improved Cyclegan-based Non-parallel Voice Conversion

Published in

IEEE International Conference on Acoustics, Spe...(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

CycleGAN-VC2 is proposed, which is an improved version of CycleGAN- VC incorporating three new techniques: an improved objective (two-step adversarial losses), improved generator (2-1-2D CNN), and improved discriminator (PatchGAN).

Abstract

Non-parallel voice conversion (VC) is a technique for learning the mapping from source to target speech without relying on parallel data. This is an important task, but it has been challenging due to the disadvantages of the training conditions. Recently, CycleGAN-VC has provided a breakthrough and performed comparably to a parallel VC method without relying on any extra data, modules, or time alignment procedures. However, there is still a large gap between the real target and converted speech, and bridging this gap remains a challenge. To reduce the gap, we propose CycleGAN-VC2, which is an improved version of CycleGAN-VC incorporating three new techniques: an improved objective (two-step adversarial losses), improved generator (2-1-2D CNN), and improved discriminator (PatchGAN). We evaluated our method on a non-parallel VC task and analyzed the effect of each technique in detail. An objective evaluation showed that these techniques help bring the converted feature sequence closer to the target in terms of both global and local structures, which we assess by using Mel-cepstral distortion and modulation spectra distance, respectively. A subjective evaluation showed that CycleGAN-VC2 outperforms CycleGAN-VC in terms of naturalness and similarity for every speaker pair, including intra-gender and inter-gender pairs.1

Authors

Takuhiro Kaneko

3 papers

H. Kameoka

3 papers

Kou Tanaka

2 papers

References59 items

ATTS2S-VC: Sequence-to-sequence Voice Conversion with Attention and Context Preservation Mechanisms

ConvS2S-VC: Fully convolutional sequence-to-sequence voice conversion

Synthetic-to-Natural Speech Waveform Conversion Using Cycle-Consistent Adversarial Networks

CycleGAN-VC: Non-parallel Voice Conversion Using Cycle-Consistent Adversarial Networks

ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder

Cyclegan-VC2: Improved Cyclegan-based Non-parallel Voice Conversion

Published in

IEEE International Conference on Acoustics, Spe...(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Takuhiro Kaneko

3 papers

H. Kameoka

3 papers

Kou Tanaka

2 papers

References59 items

ATTS2S-VC: Sequence-to-sequence Voice Conversion with Attention and Context Preservation Mechanisms

ConvS2S-VC: Fully convolutional sequence-to-sequence voice conversion

Synthetic-to-Natural Speech Waveform Conversion Using Cycle-Consistent Adversarial Networks

CycleGAN-VC: Non-parallel Voice Conversion Using Cycle-Consistent Adversarial Networks

ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder

Nobukatsu Hojo

2 papers

StarGAN-VC: non-parallel many-to-many Voice Conversion Using Star Generative Adversarial Networks

The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods

High-Quality Nonparallel Voice Conversion Based on Cycle-Consistent Adversarial Network

Non-Parallel Voice Conversion Using Variational Autoencoders Conditioned by Phonetic Posteriorgrams and D-Vectors

Non-native speech conversion with consistency-aware recursive network and generative adversarial network

Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks

Sequence-to-Sequence Voice Conversion with Similarity Metric Learned Using Generative Adversarial Networks

Generative Adversarial Network-Based Postfilter for STFT Spectrograms

Speaker-Dependent WaveNet Vocoder

DualGAN: Unsupervised Dual Learning for Image-to-Image Translation

Voice Conversion from Unaligned Corpora Using Variational Autoencoding Wasserstein Generative Adversarial Networks

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

Learning to Discover Cross-Domain Relations with Generative Adversarial Networks

Non-native speech conversion with consistency-aware recursive network and generative adversarial network

Language Modeling with Gated Convolutional Networks

Generative adversarial network-based postfilter for statistical parametric speech synthesis

F0 transformation techniques for statistical voice conversion with direct waveform modification with spectral differential

Image-to-Image Translation with Conditional Adversarial Networks

Least Squares Generative Adversarial Networks

Unsupervised Cross-Domain Image Generation

Non-Parallel Training in Voice Conversion Using an Adaptive Restricted Boltzmann Machine

WaveNet: A Generative Model for Raw Audio

A KL Divergence and DNN-Based Approach to Voice Conversion without Parallel Training Sentences

Instance Normalization: The Missing Ingredient for Fast Stylization

WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Learning Dense Correspondence via 3D-Guided Cycle Consistency

Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

Deep Residual Learning for Image Recognition

Voice conversion using deep Bidirectional Long Short-Term Memory based Recurrent Neural Networks

Adam: A Method for Stochastic Optimization

Voice Conversion Using Deep Neural Networks With Layer-Wise Generative Training

Fully convolutional networks for semantic segmentation

Exemplar-Based Sparse Representation With Residual Compensation for Voice Conversion

Voice Conversion Based on Speaker-Dependent Restricted Boltzmann Machines

Voice conversion using deep neural networks with speaker-independent pre-training

Exemplar-Based Voice Conversion Using Sparse Representation in Noisy Environments

Statistical Voice Conversion Techniques for Body-Conducted Unvoiced Speech Enhancement

Spectral Mapping Using Artificial Neural Networks for Voice Conversion

Voice Conversion Using Partial Least Squares Regression

Data-driven emotion conversion in spoken English

Low-delay voice conversion based on maximum likelihood estimation of spectral parameter trajectory

Improving the intelligibility of dysarthric speech

High Quality Voice Conversion through Phoneme-Based Linear Mapping Functions with STRAIGHT for Mandarin

Nonparallel training for voice conversion based on a parameter adaptation approach

Eigenvoice conversion based on Gaussian mixture model

Continuous probabilistic transform for voice conversion

GENERATIVE ADVERSARIAL NETS

High-order sequence modeling using speaker-dependent recurrent temporal restricted boltzmann machines for voice conversion

Speaking-aid systems using GMM-based voice conversion for electrolaryngeal speech

One-to-Many Voice Conversion Based on Tensor Representation of Speaker Space

Map-based adaptation for speech conversion using adaptation data selection and non-parallel training

, Yamato Ohtani , and Kiyohiro Shikano , “ Eigenvoice conversion based on Gaussian mixture model , ” in Proc . Interspeech ,

, Tomoki Hayashi , Kazuhiro Kobayashi , Kazuya Takeda , and Tomoki Toda , “ Speaker - dependent WaveNet vocoder , ” in

Field of Study

Computer ScienceEngineeringMathematics

Journal Information

Name

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Page

381-385

Venue Information

Name

IEEE International Conference on Acoustics, Speech, and Signal Processing

Type

conference

URL

http://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000002

Alternate Names

Int Conf Acoust Speech Signal Process
IEEE Int Conf Acoust Speech Signal Process
ICASSP
International Conference on Acoustics, Speech, and Signal Processing