DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction

Published in

IEEE International Conference on Acoustics, Spe...(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

DPM-TSE is introduced, a generative method based on diffusion probabilistic modeling (DPM) for Target Sound Extraction (TSE), to achieve both cleaner target renderings as well as improved separability from unwanted sounds.

Abstract

Common target sound extraction (TSE) approaches primarily relied on discriminative approaches in order to separate the target sound while minimizing interference from the unwanted sources, with varying success in separating the target from the background. This study introduces DPM-TSE, a generative method based on diffusion probabilistic modeling (DPM) for Target Sound Extraction (TSE), to achieve both cleaner target renderings as well as improved separability from unwanted sounds. The technique also tackles the noise floor of DPM by introducing a correction method for noise schedules and sample steps. This approach is evaluated using both objective and subjective quality metrics on the FSD Kaggle 2018 dataset. The results show that DPM-TSE has a significant improvement in perceived quality in terms of target extraction and purity.

Authors

Jiarui Hai

2 papers

Karan Thakkar

2 papers

Mounya Elhilali

2 papers

References35 items

Conditional Diffusion Model for Target Speaker Extraction

Target Speech Extraction with Conditional Diffusion Model

Common Diffusion Noise Schedules and Sample Steps are Flawed

AudioLDM: Text-to-Audio Generation with Latent Diffusion Models

Real-Time Target Sound Extraction

DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction

Published in

IEEE International Conference on Acoustics, Spe...(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Jiarui Hai

2 papers

Karan Thakkar

2 papers

Mounya Elhilali

2 papers

References35 items

Conditional Diffusion Model for Target Speaker Extraction

Target Speech Extraction with Conditional Diffusion Model

Common Diffusion Noise Schedules and Sample Steps are Flawed

AudioLDM: Text-to-Audio Generation with Latent Diffusion Models

Real-Time Target Sound Extraction

Dongchao Yang

1 papers

Diffusion-Based Generative Speech Source Separation

Speech Enhancement and Dereverberation With Diffusion-Based Generative Models

Improving Target Sound Extraction with Timestamp Information

Conditional Diffusion Probabilistic Model for Speech Enhancement

Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme

Universal Speaker Extraction in the Presence and Absence of Target Speakers for Speech of One and Two Talkers

Sound Event Detection: A tutorial

Few-shot learning of new sound classes for target sound extraction

DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism

CDPAM: Contrastive Learning for Perceptual Audio Similarity

One-Shot Conditional Audio Filtering of Arbitrary Sounds

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

WaveGrad: Estimating Gradients for Waveform Generation

Listen to What You Want: Neural Network-based Universal Sound Selector

Learning to Separate Sounds from Weakly Labeled Scenes

ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric

Conditioned Source Separation for Musical Instrument Performances

Source Separation with Weakly Labelled Data: an Approach to Computational Auditory Scene Analysis

Environmental Sound Classification with Parallel Temporal-Spectral Attention

SDR – Half-baked or Well Done?

VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking

Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation

A multi-device dataset for urban acoustic scene classification

Crowdsourced Pairwise-Comparison for Source Separation Evaluation

Attention is All you Need

Audio Set: An ontology and human-labeled dataset for audio events

U-Net: Convolutional Networks for Biomedical Image Segmentation

Performance measurement in blind audio source separation

Freesound Datasets: A Platform for the Creation of Open Audio Datasets

Field of Study

Computer ScienceEngineering

Journal Information

Name

ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Page

3920-3924

Venue Information

Name

IEEE International Conference on Acoustics, Speech, and Signal Processing

Type

conference

URL

http://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000002

Alternate Names

Int Conf Acoust Speech Signal Process
IEEE Int Conf Acoust Speech Signal Process
ICASSP
International Conference on Acoustics, Speech, and Signal Processing

TL;DR

Abstract

Authors

References35 items

Conditional Diffusion Model for Target Speaker Extraction

Target Speech Extraction with Conditional Diffusion Model

Common Diffusion Noise Schedules and Sample Steps are Flawed

AudioLDM: Text-to-Audio Generation with Latent Diffusion Models

Real-Time Target Sound Extraction

TL;DR

Abstract

Authors

References35 items

Conditional Diffusion Model for Target Speaker Extraction

Target Speech Extraction with Conditional Diffusion Model

Common Diffusion Noise Schedules and Sample Steps are Flawed

AudioLDM: Text-to-Audio Generation with Latent Diffusion Models

Real-Time Target Sound Extraction

Diffusion-Based Generative Speech Source Separation

Speech Enhancement and Dereverberation With Diffusion-Based Generative Models

Improving Target Sound Extraction with Timestamp Information

Conditional Diffusion Probabilistic Model for Speech Enhancement

Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme

Universal Speaker Extraction in the Presence and Absence of Target Speakers for Speech of One and Two Talkers

Sound Event Detection: A tutorial

Few-shot learning of new sound classes for target sound extraction

DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism

CDPAM: Contrastive Learning for Perceptual Audio Similarity

One-Shot Conditional Audio Filtering of Arbitrary Sounds

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

WaveGrad: Estimating Gradients for Waveform Generation

Listen to What You Want: Neural Network-based Universal Sound Selector

Learning to Separate Sounds from Weakly Labeled Scenes

ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric

Conditioned Source Separation for Musical Instrument Performances

Source Separation with Weakly Labelled Data: an Approach to Computational Auditory Scene Analysis

Environmental Sound Classification with Parallel Temporal-Spectral Attention

SDR – Half-baked or Well Done?

VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking

Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation

A multi-device dataset for urban acoustic scene classification

Crowdsourced Pairwise-Comparison for Source Separation Evaluation

Attention is All you Need

Audio Set: An ontology and human-labeled dataset for audio events

U-Net: Convolutional Networks for Biomedical Image Segmentation

The Creation

Performance measurement in blind audio source separation

Freesound Datasets: A Platform for the Creation of Open Audio Datasets

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names