Social Processes: Self-Supervised Meta-Learning over Conversational Groups for Forecasting Nonverbal Social Cues

Abstract

The default paradigm in the forecasting of human behavior in social conversations, involves selecting specific future semantic events of interest (e.g. speaker turn changes, group leaving) and then identifying their relationships to low-level nonverbal cues. A common hurdle in such top-down approaches is the limited availability of event-labeled data for supervised learning, stemming from the infrequency of such events. To tackle this challenge, we propose to cast forecasting into a novel bottom-up self-supervised problem to leverage the larger amount of low-level behavior cues. We formalize the task of Social Cue Forecasting (SCF), and characterize the specific modeling challenges involved. To address these we build upon key observations from social science literature and propose the Social Process (SP) models--socially aware sequence-to-sequence models that view each conversation group as a meta-learning task to account for group-specific dynamics. Our SP models learn event agnostic representations of future cues for each participant, while capturing global uncertainty by jointly reasoning about the future for all members of the group. For this novel task of SCF, improved empirical performance over non meta-learning models on real-world behavior data validates our meta-learning approach. Moreover, ablations and comparison against meta-learning models with similar assumptions validate our specific modeling choices for this task.