Home Research Papers Datasets State of the Art Pricing

Discover, visualize, and connect AI research papers. Explore the latest trends and insights in artificial intelligence research.

Product

Home
Research Papers
About

Support

Contact
Terms of Service
Privacy Policy

audio-4

Target Sound Extraction

3260 papers • 126 benchmarks • 313 datasets

Target Sound Extraction is the task of extracting a sound corresponding to a given class from an audio mixture. The audio mixture may contain background noise with a relatively low amplitude compared to the foreground mixture components. The choice of the sound class is provided as input to the model in form of a string, integer, or a one-hot encoding of the sound class.

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in target-sound-extraction-4

Trend

Dataset

Best Model

Actions

FSDSoundScapes

AudioCaps

AudioSet

Libraries

Use these libraries to find target-sound-extraction-4 models and implementations

Datasets

AudioSet

AudioCaps

FSDSoundScapes

Subtasks

Streaming Target Sound Extraction

Most implemented papers

Real-Time Target Sound Extraction

Takuya Yoshioka, Bandhav Veluri, Justin Chan, M. Itani, Tuochao Chen, Shyamnath Gollakota•Thu Nov 03 2022

Waveformer, an encoder-decoder architecture with a stack of dilated causal convolution layers as the encoder, and a transformer decoder layer as the decoder, is presented, the first neural network model to achieve real-time and streaming target sound extraction.

Content

Introduction Benchmarks Datasets Subtasks Libraries Papers

Paper Graph

Target Sound Extraction with Variable Cross-Modality Clues

Michael Zeng, Shujie Liu, Zhuo Chen, Takuya Yoshioka, Yao Qian, Y. Qian, Chenda Li, Dongmei Wang•Tue Mar 14 2023

Experimental results for seen and unseen target-sound evaluation sets show that the proposed TSE model can effectively deal with a varying number of clues which improves the TSE performance and robustness against partially compromised clues.

20 0

Paper Graph

DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction

Jiarui Hai, Karan Thakkar, Mounya Elhilali, N. Dehak, Helin Wang, Dongchao Yang•Thu Oct 05 2023

DPM-TSE is introduced, a generative method based on diffusion probabilistic modeling (DPM) for Target Sound Extraction (TSE), to achieve both cleaner target renderings as well as improved separability from unwanted sounds.

16 0

Paper Graph

Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables

Bandhav Veluri, Justin Chan, Shyamnath Gollakota, Malek Itani, Takuya Yoshioka•Sat Oct 28 2023

Imagine being able to listen to the birds chirping in a park without hearing the chatter from other hikers, or being able to block out traffic noise on a busy street while still being able to hear emergency sirens and car honks. We introduce semantic hearing, a novel capability for hearable devices that enables them to, in real-time, focus on, or ignore, specific sounds from real-world environments, while also preserving the spatial cues. To achieve this, we make two technical contributions: 1) we present the first neural network that can achieve binaural target sound extraction in the presence of interfering sounds and background noise, and 2) we design a training methodology that allows our system to generalize to real-world use. Results show that our system can operate with 20 sound classes and that our transformer-based network has a runtime of 6.56 ms on a connected smartphone. In-the-wild evaluation with participants in previously unseen indoor and outdoor scenarios shows that our proof-of-concept system can extract the target sounds and generalize to preserve the spatial cues in its binaural output. Project page with code: https://semantichearing.cs.washington.edu

32 0

Paper Graph

CLAPSep: Leveraging Contrastive Pre-Trained Model for Multi-Modal Query-Conditioned Target Sound Extraction

Hao Ma, Zhiyuan Peng, Xu Li, Mingjie Shao, Xixin Wu, Ju Liu•Mon Feb 26 2024

This paper tailor and adapt the powerful contrastive language-audio pre-trained model (CLAP) for USS, denoted as CLAPSep, and shows superior performance and zero- and few-shot generalizability of the proposed CLAPSep with fast training convergence, surpassing previous methods by a significant margin.

22 0

Paper Graph

Adding a benchmark result helps the community track progress.

Target Sound Extraction | State-of-the-Art