Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings
TL
TL;DR
This work proposes a transfer learning method for speech emotion recognition where features extracted from pre-trained wav2vec 2.0 models are modeled using simple neural networks, showing superior performance compared to results in the literature.
Abstract
Emotion recognition datasets are relatively small, making the use of the more sophisticated deep learning approaches challenging. In this work, we propose a transfer learning method for speech emotion recognition where features extracted from pre-trained wav2vec 2.0 models are modeled using simple neural networks. We propose to combine the output of several layers from the pre-trained model using trainable weights which are learned jointly with the downstream model. Further, we compare performance using two different wav2vec 2.0 models, with and without finetuning for speech recognition. We evaluate our proposed approaches on two standard emotion databases IEMOCAP and RAVDESS, showing superior performance compared to results in the literature.
References38 items
1
Contrastive Unsupervised Learning for Speech Emotion Recognition
2
General-Purpose Speech Representation Learning through a Self-Supervised Multi-Granularity Framework
3
On the Use of Self-Supervised Pre-Trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognition
4
Recognizing More Emotions with Less Data Using Self-supervised Transfer Learning
5