Emotion Recognition in Audio and Video Using Deep Neural Networks
Published in
arXiv.org(2020)External Links:
TL
TL;DR
This work attempts to explore different neural networks to improve accuracy of emotion recognition and finds (CNN+RNN) + 3DCNN multi-model architecture which processes audio spectrograms and corresponding video frames giving emotion prediction accuracy of 54.0% among 4 emotions and 71.75% among 3 emotions using IEMOCAP[2] dataset.
Abstract
Humans are able to comprehend information from multiple domains for e.g. speech, text and visual. With advancement of deep learning technology there has been significant improvement of speech recognition. Recognizing emotion from speech is important aspect and with deep learning technology emotion recognition has improved in accuracy and latency. There are still many challenges to improve accuracy. In this work, we attempt to explore different neural networks to improve accuracy of emotion recognition. With different architectures explored, we find (CNN+RNN) + 3DCNN multi-model architecture which processes audio spectrograms and corresponding video frames giving emotion prediction accuracy of 54.0% among 4 emotions and 71.75% among 3 emotions using IEMOCAP[2] dataset.
References15 items
1
Multimodal Speech Emotion Recognition and Ambiguity Resolution
2
Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network
3
Efficient Emotion Recognition from Speech Using Deep Learning on Spectrograms
4
Coupled 3D Convolutional Neural Networks for Audio-Visual Recognition
5