Multi-Person Extreme Motion Prediction (2021-05-18T00:00:00.000000Z)

TL;DR

A novel cross interaction attention mechanism that exploits historical information of both persons, and learns to predict cross dependencies between the two pose sequences to predict the future motion of two interacted persons given two sequences of their past skeletons.

Abstract

Human motion prediction aims to forecast future poses given a sequence of past 3D skeletons. While this problem has recently received increasing attention, it has mostly been tackled for single humans in isolation. In this paper, we explore this problem when dealing with humans performing collaborative tasks, we seek to predict the future motion of two interacted persons given two sequences of their past skeletons. We propose a novel cross interaction attention mechanism that exploits historical information of both persons, and learns to predict cross dependencies between the two pose sequences. Since no dataset to train such interactive situations is available, we collected ExPI (Extreme Pose Interaction) dataset, a new lab-based per-son interaction dataset of professional dancers performing Lindy-hop dancing actions, which contains 115 sequences with 30K frames annotated with 3D body poses and shapes. We thoroughly evaluate our cross interaction network on ExPI and show that both in short- and long-term predictions, it consistently outperforms state-of-the-art methods for single-person motion prediction. Our code and dataset are available at: https://team.inria.fr/robotlearn/multi-person-extreme-motionprediction/

Authors

F. Moreno-Noguer

13 papers

Xavier Alameda-Pineda

4 papers

Wen Guo

1 papers

TL;DR

Abstract

Authors

References82 items

Skeleton Graph Scattering Networks for 3D Skeleton-based Human Motion Prediction

A Unified 3D Human Motion Synthesis Model via Conditional Variational Auto-Encoder∗

Space-Time-Separable Graph Convolutional Network for Pose Forecasting

Generating Smooth Pose Sequences for Diverse Human Motion Prediction

MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction

RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting

Action-Conditioned 3D Human Motion Synthesis with Transformer VAE

TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild

Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking

Contextually Plausible and Diverse 3D Human Motion Prediction

Predicting Human Mobility with Federated Learning

PI-Net: Pose Interacting Network for Multi-Person Monocular 3D Pose Estimation

HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimation

History Repeats Itself: Human Motion Prediction via Motion Attention

Socially and Contextually Aware Human Motion and Pose Forecasting

Long-term Human Motion Prediction with Scene Context

Coherent Reconstruction of Multiple Humans From a Single Image

Learning Dynamic Relationships for 3D Human Motion Prediction

Three-Dimensional Reconstruction of Human Interactions

A Stochastic Conditioning Scheme for Diverse Human Motion Prediction

A Spatio-temporal Transformer for 3D Human Motion Prediction

DLow: Diversifying Latent Flows for Diverse Human Motion Prediction

AnimePose: Multi-person 3D pose estimation and animation

STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction

Multi-Person 3D Human Pose Estimation from Monocular Images

Deep, Robust and Single Shot 3D Multi-Person Human Pose Estimation from Monocular Images

Resolving 3D Human Pose Ambiguities With 3D Scene Constraints

Learning Trajectory Dependencies for Human Motion Prediction

Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image

Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks

XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera

Towards Natural and Accurate Future Motion Prediction of Humans and Animals

Geometric Pose Affordance: 3D Human Pose with Scene Constraints

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding

Context-Aware Human Motion Prediction

AMASS: Archive of Motion Capture As Surface Shapes

Instance- and Category-level 6D Object Pose Estimation

Human Motion Prediction via Spatio-Temporal Inpainting

Transferable Interactiveness Prior for Human-Object Interaction Detection

Action-Agnostic Human Pose Forecasting

Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera

Where Will They Go? Predicting Fine-Grained Adversarial Multi-agent Motion Using Conditional Variational Autoencoders

Adversarial Geometry-Aware Human Motion Prediction

MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics

Estimation

Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints

Anticipating Many Futures: Online Human Motion Prediction and Generation for Human-Robot Interaction

Convolutional Sequence to Sequence Model for Human Dynamics

LCR-Net++: Multi-Person 2D and 3D Pose Detection in Natural Images

Single-Shot Multi-person 3D Pose Estimation from Monocular RGB

HP-GAN: Probabilistic 3D Human Motion Prediction via GAN

Graph Attention Networks

PoseTrack: A Benchmark for Human Pose Estimation and Tracking

Social Attention: Modeling Attention in Human Crowds

LCR-Net: Localization-Classification-Regression for Human Pose

Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis

Attention is All you Need

On Human Motion Prediction Using Recurrent Neural Networks

Learning Human Motion Models for Long-Term Predictions

Deep Representation Learning for Human Motion Prediction and Classification

3D Human Pose Estimation from a Single Image via Distance Matrix Regression

Semi-Supervised Classification with Graph Convolutional Networks

Social LSTM: Human Trajectory Prediction in Crowded Spaces

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis

Structural-RNN: Deep Learning on Spatio-Temporal Graphs

Learning motion manifolds with convolutional autoencoders

Recurrent Network Models for Human Dynamics

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

Why Study the Lindy Hop?

Generalized procrustes analysis

Learning Progressive Joint Propagation for Human Motion Prediction

Cmu graphics lab motion capture database

graphics lab

L-relbow

L-lshoulder' (5) 'L-rshoulder