Long-Term Feature Banks for Detailed Video Understanding (2018-12-12T00:00:00.000000Z)

TL;DR

This paper proposes a long-term feature bank—supportive information extracted over the entire span of a video—to augment state-of-the-art video models that otherwise would only view short clips of 2-5 seconds.

Abstract

To understand the world, we humans constantly need to relate the present to the past, and put events in context. In this paper, we enable existing video models to do the same. We propose a long-term feature bank—supportive information extracted over the entire span of a video—to augment state-of-the-art video models that otherwise would only view short clips of 2-5 seconds. Our experiments demonstrate that augmenting 3D convolutional networks with a long-term feature bank yields state-of-the-art results on three challenging video datasets: AVA, EPIC-Kitchens, and Charades. Code is available online.

Authors

Kaiming He

22 papers

Philipp Krähenbühl

10 papers

Ross B. Girshick

23 papers

TL;DR

Abstract

Authors

References62 items

Scaling Egocentric Vision: The Dataset

Recurrent Tubelet Proposal and Recognition Networks for Action Detection

Non-local NetVLAD Encoding for Video Classification

Actor-Centric Relation Network

A Better Baseline for AVA

Object Level Visual Reasoning in Videos

Videos as Space-Time Region Graphs

Scaling Egocentric Vision: The EPIC-KITCHENS Dataset

Rethinking Spatiotemporal Feature Learning For Video Understanding

Compressed Video Action Recognition

Temporal Relational Reasoning in Videos

Non-local Neural Networks

Attend and Interact: Higher-Order Object Interactions for Video Understanding

Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks

ConvNet Architecture Search for Spatiotemporal Feature Learning

Lattice Long Short-Term Memory for Human Action Recognition

Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering

Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding

Learnable pooling with Context Gating for video classification

Attention is All you Need

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

Action Tubelet Detector for Spatio-Temporal Action Localization

AMTnet: Action-Micro-Tube Regression by End-to-end Trainable Deep Architecture

Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos

Mask R-CNN

Asynchronous Temporal Fields for Action Recognition

Feature Pyramid Networks for Object Detection

Online Real-Time Multiple Spatiotemporal Action Localisation and Prediction

Aggregated Residual Transformations for Deep Neural Networks

Spatiotemporal Residual Networks for Video Action Recognition

Multi-region Two-Stream R-CNN for Action Detection

Temporal Segment Networks: Towards Good Practices for Deep Action Recognition

Layer Normalization

VideoLSTM convolves, attends and flows for action recognition

Long-Term Temporal Convolutions for Action Recognition

Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding

Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations

Deep Residual Learning for Image Recognition

Learning to Track for Spatio-Temporal Action Localization

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Fast R-CNN

End-To-End Memory Networks

Beyond short snippets: Deep networks for video classification

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Learning Spatiotemporal Features with 3D Convolutional Networks

Finding action tubes

Long-term recurrent convolutional networks for visual recognition and description

ImageNet Large Scale Visual Recognition Challenge

Large-Scale Video Classification with Convolutional Neural Networks

Two-Stream Convolutional Networks for Action Recognition in Videos

Microsoft COCO: Common Objects in Context

Detection bank: an object detection based video representation for multimedia event recognition

Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification

Et al

Human Centric Spatio-Temporal Action Localization

and K

Dropout: a simple way to prevent neural networks from overfitting

Quo vadis?

Learning Process in an Asymmetric Threshold Network

Quo Vadis

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names