Comparison of Spatio-Temporal Models for Human Motion and Pose Forecasting in Face-to-Face Interaction Scenarios (2022-03-07T00:00:00.000000Z)

TL;DR

This work presents the first systematic comparison of state-of-the-art approaches for behavior forecasting with whole-body annotations from the very recently released UDIVA v0.5, and shows that by autoregressively predicting the future with methods trained for the short-term future, this finding holds when highly noisy annotations are used, which opens new horizons towards the use of weak-supervised learning.

Abstract

Human behavior forecasting during human-human interactions is of utmost importance to provide robotic or virtual agents with social intelligence. This problem is especially challenging for scenarios that are highly driven by interpersonal dynamics. In this work, we present the first systematic comparison of state-of-the-art approaches for behavior forecasting. To do so, we leverage whole-body annotations (face, body, and hands) from the very recently released UDIVA v0.5, which features face-to-face dyadic interactions. Our best attention-based approaches achieve state-of-the-art performance in UDIVA v0.5. We show that by autoregressively predicting the future with methods trained for the short-term future (<400ms), we outperform the baselines even for a considerably longer-term future (up to 2s). We also show that this finding holds when highly noisy annotations are used, which opens new horizons towards the use of weakly-supervised learning. Combined with large-scale datasets, this may help boost the advances in this field.

Authors

Zhen Xu

2 papers

Isabelle M Guyon

8 papers

Sergio Escalera

31 papers

TL;DR

Abstract

Authors

References62 items

Didn't see that coming: a survey on non-verbal social human behavior forecasting

Dyadic Human Motion Prediction

Multi-Person 3D Motion Prediction with Multi-Range Transformers

Forecasting Nonverbal Social Signals during Dyadic Interactions with Generative Adversarial Neural Networks

Simple Baseline for Single Human Motion Forecasting

Fast User Adaptation for Human Motion Prediction in Physical Human–Robot Interaction

A Systematic Review of Human and Robot Personality in Health Care Human-Robot Interaction

Multi-level Motion Attention for Human Motion Prediction

Politics of technology: the use of artificial intelligence by US and Canadian immigration agencies and their impacts on human rights

TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild

Trajevae: Controllable Human Motion Generation from Trajectories

3D Human Pose Estimation with Spatial and Temporal Transformers

Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Dataset

Forecasting Characteristic 3D Poses of Human Actions

History Repeats Itself: Human Motion Prediction via Motion Attention

Socially and Contextually Aware Human Motion and Pose Forecasting

DeepSSM: Deep State-Space Model for 3D Human Motion Prediction

Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks

Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction

Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction

Symbiotic Graph Neural Networks for 3D Skeleton-Based Human Action Recognition and Motion Prediction

To React or not to React: End-to-End Visual Pose Forecasting for Personalized Avatar during Dyadic Conversations

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Artificial Intelligence Prediction and Counterterrorism

A Survey of Behavioral Models for Social Robots

Towards More Realistic Human-Robot Conversation: A Seq2Seq-based Body Gesture Interaction System

Context-Aware Human Motion Prediction

Deep appearance models for face rendering

The impact of pedagogical agent gesturing in multimedia learning environments: A meta-analysis

Convolutional Sequence to Sequence Model for Human Dynamics

An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

Facial expression recognition as a candidate marker for autism spectrum disorder: how frequent and severe are deficits?

Learn2Smile: Learning non-verbal interaction through observation

Attention is All you Need

On Human Motion Prediction Using Recurrent Neural Networks

Learning Human Motion Models for Long-Term Predictions

Using Facially Expressive Robots to Calibrate Clinical Pain Perception

Temporal Convolutional Networks for Action Segmentation and Detection

CNN architectures for large-scale audio classification

Social LSTM: Human Trajectory Prediction in Crowded Spaces

Big Data Recommendations for Industrial–Organizational Psychology

Look Like Me: Matching Robot Personality via Gaze to Increase Motivation

Adam: A Method for Stochastic Optimization

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Sequence to Sequence Learning with Neural Networks

Global Clustering of Countries by Culture – An Extension of the GLOBE Study

Early markers of autism spectrum disorders in infants and toddlers prospectively identified in the Social Attention and Communication Study

Human-Robot Interaction in Social Robotics

Expressive robots in education: varying the degree of social supportive behavior of a robotic tutor

Social signal processing: Survey of an emerging domain

Face to Face: The Neural Basis of Social Mirroring and Empathy

The relationship context of human behavior and development.

Long Short-Term Memory

An introduction to the five-factor model and its applications.

Social Processes: Self-Supervised Meta-Learning over Conversational Groups for Forecasting Nonverbal Social Cues

ChaLearn LAP Challenges on Self-Reported Personality Recognition and Non-Verbal Behavior Forecasting During Social Dyadic Interactions: Dataset, Design, and Results

Multi-Person Extreme Motion Prediction with Cross-Interaction Attention

RNN-based Motion Prediction in Competitive Fencing Considering Interaction between Players

Anticipation and its Applications in Human-machine Interaction

Temporal

Is walking a random walk? Evidence for long-range correlations in stride interval of human gait.

Interpersonal Adaptation: Dyadic Interaction Patterns

Field of Study