Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos (2018-06-01T00:00:00.000000Z)

TL;DR

A deep neural framework, termed Conversational Memory Network (CMN), is proposed, which leverages contextual information from the conversation history to recognize utterance-level emotions in dyadic conversational videos.

Abstract

Emotion recognition in conversations is crucial for the development of empathetic machines. Present methods mostly ignore the role of inter-speaker dependency relations while classifying emotions in conversations. In this paper, we address recognizing utterance-level emotions in dyadic conversational videos. We propose a deep neural framework, termed Conversational Memory Network (CMN), which leverages contextual information from the conversation history. In particular, CMN uses multimodal approach comprising audio, visual and textual features with gated recurrent units to model past utterances of each speaker into memories. These memories are then merged using attention-based hops to capture inter-speaker dependencies. Experiments show a significant improvement of 3 − 4% in accuracy over the state of the art.

Authors

Amir Zadeh

12 papers

Louis-philippe Morency

21 papers

E. Cambria

25 papers

TL;DR

Abstract

Authors

References54 items

Augmenting End-to-End Dialogue Systems With Commonsense Knowledge

SenticNet 5: Discovering Conceptual Primitives for Sentiment Analysis by Means of Context Embeddings

Pooling acoustic and lexical features for the prediction of valence

Multi-level Multiple Attentions for Contextual Multimodal Sentiment Analysis

Sentiment Analysis Is a Big Suitcase

Natural language based financial forecasting: a survey

Augmenting End-to-End Dialog Systems with Commonsense Knowledge

A review of affective computing: From unimodal analysis to multimodal fusion

Recent Trends in Deep Learning Based Natural Language Processing

Context-Dependent Sentiment Analysis in User-Generated Videos

Tensor Fusion Network for Multimodal Sentiment Analysis

End-to-End Multimodal Emotion Recognition Using Deep Neural Networks

Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory

Real-Time Speech Emotion and Sentiment Recognition for Interactive Dialogue Systems

A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks

Select-additive learning: Improving generalization in multimodal sentiment analysis

Mirroring Facial Expressions and Emotions in Dyadic Conversations

Multimodal emotion recognition using deep learning architectures

The Handbook of Conversation Analysis

Ask Me Anything: Dynamic Memory Networks for Natural Language Processing

End-To-End Memory Networks

A Review and Meta-Analysis of Multimodal Affect Detection Systems

Semantic Audiovisual Data Fusion for Automatic Emotion Recognition

Learning Spatiotemporal Features with 3D Convolutional Networks

Neural Turing Machines

Memory Networks

Neural Machine Translation by Jointly Learning to Align and Translate

Convolutional Neural Networks for Sentence Classification

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

A Convolutional Neural Network for Modelling Sentences

Emotional Dynamics and Strategizing Processes: A Study of Strategic Conversations in Top Team Meetings

Predicting Depression via Social Media

Emotional state transition model based on stimulus and personality characteristics

Ensemble of SVM trees for multimodal emotion recognition

Emotion, Affect and Conversation

Random Search for Hyper-Parameter Optimization

Emotion recognition using bimodal data fusion

Multimodal fusion for multimedia analysis: a survey

Opensmile: the munich versatile and fast open-source audio feature extractor

IEMOCAP: interactive emotional dyadic motion capture database

Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech

Analysis of emotion recognition using facial expressions, speech and multimodal information

Audio-visual based emotion recognition - a new approach

Emotion Regulation in Romantic Relationships: The Cognitive Consequences of Concealing Feelings

Towards an automatic classification of emotions in speech

Long Short-Term Memory

Recognizing emotion in speech

Changing emotion dynamics: individual differences in the effect of anticipatory social stress on emotional inertia.

Classifying Skewed Data: Importance Weighting to Optimize Average Recall

Text-Based Emotion Transformation Analysis

Context-sensitive multimodal emotion recognition from speech and facial expression using bidirectional LSTM modeling

Affective Computing: From Laughter to IEEE

How Emotions Work: The Social Functions of Emotional Expression in Negotiations

Aug - menting end - to - end dialog systems with common - sense knowledge . In AAAI . Tom Young , Devamanyu Hazarika , Soujanya Poria , and Erik Cambria

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names