Toward Universal Text-To-Music Retrieval (2022-11-26T00:00:00.000000Z)

TL;DR

This work reviews recent text-based music retrieval systems using a proposed benchmark in two main aspects: input text representation and training objectives and enables a universal text-to-music retrieval system that achieves comparable retrieval performances in both tag- and sentence-level inputs.

Abstract

This paper introduces effective design choices for text-to-music retrieval systems. An ideal text-based retrieval system would support various input queries such as pre-defined tags, unseen tags, and sentence-level descriptions. In reality, most previous works mainly focused on a single query type (tag or sentence) which may not generalize to another input type. Hence, we review recent text-based music retrieval systems using our proposed benchmark in two main aspects: input text representation and training objectives. Our findings enable a universal text-to-music retrieval system that achieves comparable retrieval performances in both tag- and sentence-level inputs. Furthermore, the proposed multimodal representation generalizes to 9 different downstream music classification tasks. We present the code and demo online.1

Authors

Juhan Nam

16 papers

Seungheon Doh

7 papers

Minz Won

3 papers

TL;DR

Abstract

Authors

References50 items

Supervised and Unsupervised Learning of Audio Representations for Music Understanding

MuLan: A Joint Embedding of Music Audio and Natural Language

Contrastive Audio-Language Learning for Music

Learning Music Sequence Representation From Text Supervision

Learning Music Audio Representations Via Weak Language Supervision

Emotion Embedding Spaces for Matching Music to Stories

Semi-supervised Music Tagging Transformer

Codified audio language modeling learns useful representations for music information retrieval

Learning Transferable Visual Models From Natural Language Supervision

Multimodal Metric Learning for Tag-Based Music Retrieval

Metric learning vs classification for disentangled music representation learning

Evaluation of CNN-based Automatic Music Tagging Models

Semantic Tagging of Singing Voices in Popular Music Recordings

Multi-Task Music Representation Learning from Multi-Label Embeddings

Zero-shot Learning for Audio-based Music Classification and Tagging

Zero-shot Learning and Knowledge Transfer in Music Classification and Tagging

Deep Learning for Audio-Based Music Classification and Tagging: Teaching Computers to Distinguish Rock from Bach

Representation Learning with Contrastive Predictive Coding

"Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

End-to-end Learning for Music Audio Tagging at Scale

FMA: A Dataset for Music Analysis

Understanding intermediate layers using linear classifier probes

CNN architectures for large-scale audio classification

Automatic Tagging Using Deep Convolutional Neural Networks

Learning Deep Structure-Preserving Image-Text Embeddings

GloVe: Global Vectors for Word Representation

Semantic Annotation and Retrieval of Music and Sound Effects

Lyrics-Based Audio Retrieval and Multimodal Navigation in Music Collections

Factors in automatic musical genre classification of audio signals

Music Representation Learning Based on Editorial Metadata from Discogs

Learning a cross-domain embedding space of vocal and mixed audio with a structure-preserving triplet loss

The MTG-Jamendo Dataset for Automatic Music Tagging

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Query-by-Blending: A Music Exploration System Blending Latent Vector Representations of Lyric Word, Song Audio, and Artist

hey alexa, what&#x2019;s up?

Transfer Learning by Supervised Pre-training for Audio-based Music Classification

“The gtzan dataset: Its contents, its faults, their effects on evaluation, and its future use

The million song dataset

Evaluation of Algorithms Using Games: The Case of Music Tagging

Query by humming: musical information retrieval in an audio database

Fabien

Information Processing and Management Studying emotion induced by music through a crowdsourcing game

Contrastive BERTbase Stochastic 90

contains 55,000 full-length audio tracks with 195 tags from the Jamendo music platform

only one split

contains 30-second clips from 10 distinct gen-res for a single-label multi-class classiﬁcation

A.2. Downstream Dataset For downstream evaluation, we use multiple heterogeneous datasets

experiments including a list of the datasets, models, and qualitative and quantitative results. A.1

CALS subset with 500 Last.fm tags [11] and 1,402

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names

hey alexa, what’s up?