WGANSing: A Multi-Voice Singing Voice Synthesizer Based on the Wasserstein-GAN (2019-03-26T00:00:00.000000Z)

TL;DR

A deep neural network based singing voice synthesizer, inspired by the Deep Convolutions Generative Adversarial Networks (DCGAN) architecture and optimized using the Wasserstein-GAN algorithm, which facilitates the modelling of the large variability of pitch in the singing voice.

Abstract

We present a deep neural network based singing voice synthesizer, inspired by the Deep Convolutions Generative Adversarial Networks (DCGAN) architecture and optimized using the Wasserstein-GAN algorithm. We use vocoder parameters for acoustic modelling, to separate the influence of pitch and timbre. This facilitates the modelling of the large variability of pitch in the singing voice. Our network takes a block of consecutive frame-wise linguistic and fundamental frequency features, along with global singer identity as input and outputs vocoder features, corresponding to the block of features. This block-wise approach, along with the training methodology allows us to model temporal dependencies within the features of the input block. For inference, sequential blocks are concatenated using an overlap-add procedure. We show that the performance of our model is competitive with regards to the state-of-the-art and the original sample using objective metrics and a subjective listening test. We also present examples of the synthesis on a supplementary website and the source code via GitHub.

Authors

Pritish Chandna

2 papers

Merlijn Blaauw

1 papers

J. Bonada

1 papers

WGANSing: A Multi-Voice Singing Voice Synthesizer Based on the Wasserstein-GAN

TL;DR

Abstract

Authors

References23 items

Generative Adversarial Networks

Singing Voice Synthesis Based on Generative Adversarial Networks

Data Efficient Voice Cloning for Neural Singing Synthesis

GANSynth: Adversarial Neural Audio Synthesis

Harmonizing Maximum Likelihood with GANs for Multimodal Conditional Generation

Wasserstein GAN and Waveform Loss-Based Acoustic Model Training for Multi-Speaker Text-to-Speech Synthesis Systems Using a WaveNet Vocoder

Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation

Adversarial Audio Synthesis

Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

Generative Adversarial Network-Based Postfilter for STFT Spectrograms

Wasserstein Generative Adversarial Networks

Statistical parametric speech synthesis using generative adversarial networks under a multi-task learning framework

A Neural Parametric Singing Synthesizer

Monoaural Audio Source Separation Using Deep Convolutional Neural Networks

Generative adversarial network-based postfilter for statistical parametric speech synthesis

WaveNet: A Generative Model for Raw Audio

WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications

Improved Techniques for Training GANs

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

Conditional Generative Adversarial Nets

The NUS sung and spoken lyrics corpus: A quantitative comparison of singing and speech

A GENERATIVE ADVERSARIAL NETWORK FOR STYLE MODELING IN A TEXT-TO-SPEECH SYSTEM

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names