Encouraging LSTMs to Anticipate Actions Very Early
TL
TL;DR
A new action anticipation method that achieves high prediction accuracy even in the presence of a very small percentage of a video sequence, and develops a multi-stage LSTM architecture that leverages context-aware and action-aware features, and introduces a novel loss function that encourages the model to predict the correct class as early as possible.
Abstract
In contrast to the widely studied problem of recognizing an action given a complete sequence, action anticipation aims to identify the action from only partially available videos. As such, it is therefore key to the success of computer vision applications requiring to react as early as possible, such as autonomous navigation. In this paper, we propose a new action anticipation method that achieves high prediction accuracy even in the presence of a very small percentage of a video sequence. To this end, we develop a multi-stage LSTM architecture that leverages context-aware and action-aware features, and introduce a novel loss function that encourages the model to predict the correct class as early as possible. Our experiments on standard benchmark datasets evidence the benefits of our approach; We outperform the state-of-the-art action anticipation methods for early prediction by a relative increase in accuracy of 22.0% on JHMDB-21, 14.0% on UT-Interaction and 49.9% on UCF-101.
References58 items
1
Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
2
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
3
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
4
Learning Activity Progression in LSTMs for Activity Detection and Early Detection
5