Music Mood Detection Based on Audio and Lyrics with Deep Neural Net
Published in
International Society for Music Information Ret...(2018)External Links:
TL
TL;DR
This work reproduces the implementation of traditional feature engineering based approaches and proposes a new model based on deep learning that outperforms classical models on the arousal detection task, and shows that both approaches perform equally on the valence prediction task.
Abstract
We consider the task of multimodal music mood prediction based on the audio signal and the lyrics of a track. We reproduce the implementation of traditional feature engineering based approaches and propose a new model based on deep learning. We compare the performance of both approaches on a database containing 18,000 tracks with associated valence and arousal values and show that our approach outperforms classical models on the arousal detection task, and that both approaches perform equally on the valence prediction task. We also compare the a posteriori fusion with fusion of modalities optimized simultaneously with each unimodal model, and observe a significant improvement of valence prediction. We release part of our database for comparison purposes.
References32 items
1
A framework for evaluating multimodal music mood classification
2
Graph-Based Multimodal Music Mood Classification in Discriminative Latent Space
3
Bi-Modal Deep Boltzmann Machine Based Musical Emotion Classification
4
Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
5