Multi-Modal Sarcasm Detection and Humor Classification in Code-Mixed Conversations

Abstract

Sarcasm detection and humor classification are inherently subtle problems, primarily due to their dependence on the contextual and non-verbal information. Furthermore, existing studies in these two topics are usually constrained in non-English languages such as Hindi, due to the unavailability of qualitative annotated datasets. In this work, we make two major contributions considering the above limitations: (1) we develop a Hindi-English code-mixed dataset, <monospace>MaSaC</monospace>,¹ for the multi-modal sarcasm detection and humor classification in conversational dialog, which to our knowledge is the first dataset of its kind; (2) we propose <monospace>MSH-COMICS</monospace>,² a novel attention-rich neural architecture for the utterance classification. We learn efficient utterance representation utilizing a hierarchical attention mechanism that attends to a small portion of the input sentence at a time. Further, we incorporate dialog-level contextual attention mechanism to leverage the dialog history for the multi-modal classification. We perform extensive experiments for both the tasks by varying multi-modal inputs and various submodules of <monospace>MSH-COMICS</monospace>. We also conduct comparative analysis against existing approaches. We observe that <monospace>MSH-COMICS</monospace> attains superior performance over the existing models by <inline-formula><tex-math notation="LaTeX">$>$</tex-math><alternatives><mml:math><mml:mo>></mml:mo></mml:math><inline-graphic xlink:href="akhtar-ieq1-3083522.gif"/></alternatives></inline-formula>1 F1-score point for the sarcasm detection and 10 F1-score points in humor classification. We diagnose our model and perform thorough analysis of the results to understand the superiority and pitfalls.