Peeking Into the Future: Predicting Future Person Activities and Locations in Videos (2019-02-11T00:00:00.000000Z)

TL;DR

An end-to-end, multi-task learning system utilizing rich visual features about human behavioral information and interaction with their surroundings is proposed, providing the first empirical evidence that joint modeling of paths and activities benefits future path prediction.

Abstract

Deciphering human behaviors to predict their future paths/trajectories and what they would do from videos is important in many applications. Motivated by this idea, this paper studies predicting a pedestrian's future path jointly with future activities. We propose an end-to-end, multi-task learning system utilizing rich visual features about human behavioral information and interaction with their surroundings. To facilitate the training, the network is learned with an auxiliary task of predicting future location in which the activity will happen. Experimental results demonstrate our state-of-the-art performance over two public benchmarks on future trajectory prediction. Moreover, our method is able to produce meaningful future activity prediction in addition to the path. The result provides the first empirical evidence that joint modeling of paths and activities benefits future path prediction.

Authors

Lu Jiang

9 papers

Li Fei-Fei

37 papers

Juan Carlos Niebles

9 papers

TL;DR

Abstract

Authors

References40 items

TRECVID 2018: Benchmarking Video Activity Detection, Video Captioning and Matching, Video Storytelling Linking and Video Search

Scene-LSTM: A Model for Human Trajectory Prediction

Learning and Inferring “Dark Matter” and Predicting Human Intents and Trajectories in Videos

A Transferable Pedestrian Motion Prediction Model for Intersections with Different Geometries

SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints

Encoding Crowd Interaction with Deep Neural Network for Pedestrian Trajectory Prediction

Focal Visual-Text Attention for Visual Question Answering

Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Understanding Human Behaviors in Crowds by Imitating the Decision-Making Process

Future Person Localization in First-Person Videos

Relation Networks for Object Detection

CERN: Confidence-Energy Recurrent Network for Group Activity Recognition

Encouraging LSTMs to Anticipate Actions Very Early

Mask R-CNN

Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies

Feature Pyramid Networks for Object Detection

RMPE: Regional Multi-person Pose Estimation

Pedestrian Behavior Understanding and Prediction with Deep Neural Networks

Learning Activity Progression in LSTMs for Activity Detection and Early Detection

Social LSTM: Human Trajectory Prediction in Crowded Spaces

Structural-RNN: Deep Learning on Spatio-Temporal Graphs

Object Tracking Benchmark

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Joint inference of groups, events and human roles in aerial videos

Car that Knows Before You Do: Anticipating Maneuvers via Learning Temporal Driving Models

Unsupervised Learning of Video Representations using LSTMs

Context-Based Pedestrian Path Prediction

Understanding Collective Activitiesof People from Videos

Anticipating Human Activities Using Object Affordances for Reactive Robotic Response

ADADELTA: An Adaptive Learning Rate Method

A large-scale benchmark dataset for event recognition in surveillance video

Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings

People tracking with human motion predictions from social forces

Et al

Crowds by Example

and s

Li Fei-Fei

Research Showcase @ CMU

Social lstm: Human tra-2

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names