A Tutorial on Deep Learning for Music Information Retrieval (2017-09-13T00:00:00.000000Z)

TL;DR

The basic principles and prominent works in deep learning for MIR are laid out and the network structures that have been successful in MIR problems are outlined to facilitate the selection of building blocks for the problems at hand.

Abstract

Following their success in Computer Vision and other areas, deep learning techniques have recently become widely adopted in Music Information Retrieval (MIR) research. However, the majority of works aim to adopt and assess methods that have been shown to be effective in other domains, while there is still a great need for more original research focusing on music primarily and utilising musical knowledge and insight. The goal of this paper is to boost the interest of beginners by providing a comprehensive tutorial and reducing the barriers to entry into deep learning for MIR. We lay out the basic principles and review prominent works in this hard to navigate field. We then outline the network structures that have been successful in MIR problems and facilitate the selection of building blocks for the problems at hand. Finally, guidelines for new tasks and some advanced topics in deep learning are discussed to stimulate new research in this fascinating field.

Authors

Kyunghyun Cho

23 papers

Keunwoo Choi

8 papers

György Fazekas

6 papers

TL;DR

Abstract

Authors

References127 items

A Comparison of Audio Signal Preprocessing Methods for Deep Neural Networks on Music Tagging

Kapre: On-GPU Audio Preprocessing Layers for a Quick Implementation of Deep Neural Network Models with Keras

The Effects of Noisy Labels on Deep Convolutional Neural Networks for Music Tagging

Stacked Convolutional and Recurrent Neural Networks for Music Emotion Recognition

The Effects of Noisy Labels on Deep Convolutional Neural Networks for Music Classification

Deep Cross-Modal Audio-Visual Generation

A Neural Parametric Singing Synthesizer

Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders

BEGAN: Boundary Equilibrium Generative Adversarial Networks

MidiNet: A Convolutional Generative Adversarial Network for Symbolic-Domain Music Generation

SEGAN: Speech Enhancement Generative Adversarial Network

Transfer Learning for Music Classification and Regression Tasks

Timbre analysis of music audio signals with convolutional neural networks

Sample-level Deep Convolutional Neural Networks for Music Auto-tagging Using Raw Waveforms

Multi-Level and Multi-Scale Feature Aggregation Using Pretrained Convolutional Neural Networks for Music Auto-Tagging

Designing efficient architectures for modeling temporal features with convolutional neural networks

Revisiting the problem of audio-based hit song prediction using convolutional neural networks

Deep ranking: Triplet MatchNet for music metric learning

Wasserstein GAN

Feature Learning for Chord Recognition: The Deep Chroma Extractor

On the Potential of Simple Framewise Approaches to Piano Transcription

SampleRNN: An Unconditional End-to-End Neural Audio Generation Model

SoundNet: Learning Sound Representations from Unlabeled Video

An overview of gradient descent optimization algorithms

Convolutional recurrent neural networks for music classification

WaveNet: A Generative Model for Raw Audio

A fully convolutional deep auditory model for musical chord recognition

Learning high-level features for chord recognition using Autoencoder

Explaining Deep Convolutional Neural Networks on Music Classification

Automatic Tagging Using Deep Convolutional Neural Networks

Deep Convolutional Neural Networks for Predominant Instrument Recognition in Polyphonic Music

Deep Convolutional Networks on the Pitch Spiral For Music Instrument Recognition

A deep bidirectional long short-term memory based multi-scale approach for music dynamic emotion prediction

Automatic Chord estimation on seventhsbass Chord vocabulary using deep neural network

Audio Word2Vec: Unsupervised Learning of Audio Segment Representations Using Sequence-to-Sequence Autoencoder

Mastering the game of Go with deep neural networks and tree search

Music boundary detection using neural networks on spectrograms and self-similarity lag matrices

Deep Residual Learning for Image Recognition

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Exploring Data Augmentation for Improved Singing Voice Detection with Neural Networks

A Primer on Neural Network Models for Natural Language Processing

Hierarchical Evaluation of Segment Boundary Detection

An End-to-End Neural Network for Polyphonic Piano Music Transcription

An End-to-End Neural Network for Polyphonic Music Transcription

Blocks and Fuel: Frameworks for deep learning

Highway Networks

Downbeat tracking with multiple features and deep neural networks

Deep neural network based instrument extraction from music

Singing voice detection with deep recurrent neural networks

Deep Karaoke: Extracting Vocals from Musical Mixtures Using a Convolutional Deep Neural Network

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Adam: A Method for Stochastic Optimization

A hybrid recurrent neural network for music transcription

Conditional Generative Adversarial Nets

Singing-Voice Separation from Monaural Recordings using Deep Recurrent Neural Networks

Going deeper with convolutions

Sequence to Sequence Learning with Neural Networks

Neural Machine Translation by Jointly Learning to Align and Translate

Towards time-varying music auto-tagging based on CAL500 expansion

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

End-to-end learning for music audio

Improved musical onset detection with Convolutional Neural Networks

Improved music feature learning with deep neural networks

From music audio to chord tablature: Teaching deep convolutional networks toplay guitar

Deep learning in neural networks: An overview

Network In Network

Deep content-based music recommendation

Visualizing and Understanding Convolutional Networks

Improving deep neural networks for LVCSR using rectified linear units and dropout

On rectified linear units for speech processing

Rethinking Automatic Chord Recognition with Convolutional Neural Networks

ImageNet classification with deep convolutional neural networks

Deep Sparse Rectifier Neural Networks

Simple methods for improving speaker-similarity of HMM-based speech synthesis

Melody line estimation in homophonic music audio signals based on temporal-variability of melodic source