LSTA: Long Short-Term Attention for Egocentric Action Recognition
TL
TL;DR
This paper proposes LSTA as a mechanism to focus on features from spatial relevant parts while attention is being tracked smoothly across the video sequence, achieving state-of-the-art performance on four standard benchmarks.
Abstract
Egocentric activity recognition is one of the most challenging tasks in video analysis. It requires a fine-grained discrimination of small objects and their manipulation. While some methods base on strong supervision and attention mechanisms, they are either annotation consuming or do not take spatio-temporal patterns into account. In this paper we propose LSTA as a mechanism to focus on features from spatial relevant parts while attention is being tracked smoothly across the video sequence. We demonstrate the effectiveness of LSTA on egocentric activity recognition with an end-to-end trainable two-stream architecture, achieving state-of-the-art performance on four standard benchmarks.
References48 items
1
Multi-Stream Deep Neural Networks for RGB-D Egocentric Action Recognition
2
Making Third Person Techniques Recognize First-Person Actions in Egocentric Videos
3
In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video
4
Egocentric Activity Prediction via Event Modulated Attention
5