MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement (2019-05-13T00:00:00.000000Z)

TL;DR

A novel MetricGAN approach with an aim to optimize the generator with respect to one or multiple evaluation metrics, which could not be fully optimized by Lp or conventional adversarial losses is proposed.

Abstract

Adversarial loss in a conditional generative adversarial network (GAN) is not designed to directly optimize evaluation metrics of a target task, and thus, may not always guide the generator in a GAN to generate data with improved metric scores. To overcome this issue, we propose a novel MetricGAN approach with an aim to optimize the generator with respect to one or multiple evaluation metrics. Moreover, based on MetricGAN, the metric scores of the generated data can also be arbitrarily specified by users. We tested the proposed MetricGAN on a speech enhancement task, which is particularly suitable to verify the proposed approach because there are multiple metrics measuring different aspects of speech signals. Moreover, these metrics are generally complex and could not be fully optimized by Lp or conventional adversarial losses.

Authors

Szu-Wei Fu

4 papers

Yu Tsao

4 papers

Chien-Feng Liao

2 papers

TL;DR

Abstract

Authors

References53 items

Learning With Learned Loss Function: Speech Enhancement With Quality-Net to Improve Perceptual Evaluation of Speech Quality

Sergan: Speech Enhancement Using Relativistic Generative Adversarial Networks with Gradient Penalty

Phase-aware Speech Enhancement with Deep Complex U-Net

End-to-end Networks for Supervised Single-channel Speech Separation

DNN-Based Source Enhancement to Increase Objective Sound Quality Assessment Score

Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation

A Deep Learning Loss Function Based on the Perceptual Evaluation of the Speech Quality

Error Modeling via Asymmetric Laplace Distribution for Deep Neural Network Based Single-Channel Speech Enhancement

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM

Noise Adaptive Speech Enhancement using Domain Adversarial Training

Deep Neural Network Based Speech Separation Optimizing an Objective Estimator of Intelligibility for Low Latency Applications

Speech Denoising with Deep Feature Losses

Convolutional Neural Networks to Enhance Coded Speech

Performance Based Cost Functions for End-to-End Speech Separation

A Deep Model With Local Surrogate Loss for General Cost-Sensitive Multi-Label Learning

Training Supervised Speech Separation System to Improve STOI and PESQ Directly

Time-Frequency Masking-Based Speech Enhancement Using Generative Adversarial Network

Perceptually Guided Speech Enhancement Using Deep Neural Networks

On Adversarial Training and Loss Functions for Speech Enhancement

Spectral Feature Mapping with MIMIC Loss for Robust Speech Recognition

Raw Multi-Channel Audio Source Separation using Multi- Resolution Convolutional Auto-Encoders

Spectral Normalization for Generative Adversarial Networks

Monaural Speech Enhancement Using Deep Neural Networks by Maximizing a Short-Time Objective Intelligibility Measure

Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition

End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks

Supervised Speech Separation Based on Deep Learning: An Overview

Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification

SEGAN: Speech Enhancement Generative Adversarial Network

Multichannel End-to-end Speech Recognition

DNN-based source enhancement self-optimized by reinforcement learning using sound quality measurements

Image-to-Image Translation with Conditional Adversarial Networks

Least Squares Generative Adversarial Networks

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Investigating RNN-based speech enhancement methods for noise-robust Text-to-Speech

Speech Enhancement with LSTM Recurrent Neural Networks and its Application to Noise-Robust ASR

Adam: A Method for Stochastic Optimization

On Training Targets for Supervised Speech Separation

Conditional Generative Adversarial Nets

Ideal ratio mask estimation using deep neural networks for robust speech recognition

An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy Speech

2010 Special Issue: Parameter-exploring policy gradients

Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs

Crafting Papers on Machine Learning

Speech Enhancement with a Generative Adversarial Network

GENERATIVE ADVERSARIAL NETS

A Regression Approach to Speech Enhancement Based on Deep Neural Networks

An Experimental Study on Speech Enhancement Based on Deep Neural Networks

Speech enhancement based on deep denoising autoencoder

100 nonspeech environmental sounds

Getting started with the darpa timit cd-rom: An acoustic phonetic continuous speech database

Speech enhancement

MetricGAN

Field of Study

Venue Information

Name

Type

URL

Alternate Names