Flow-Guided Feature Aggregation for Video Object Detection (2017-03-29T00:00:00.000000Z)

TL;DR

This work presents flow-guided feature aggregation, an accurate and end-to-end learning framework for video object detection that improves the per-frame features by aggregation of nearby features along the motion paths, and thus improves the video recognition accuracy.

Abstract

Extending state-of-the-art object detectors from image to video is challenging. The accuracy of detection suffers from degenerated object appearances in videos, e.g., motion blur, video defocus, rare poses, etc. Existing work attempts to exploit temporal information on box level, but such methods are not trained end-to-end. We present flow-guided feature aggregation, an accurate and end-to-end learning framework for video object detection. It leverages temporal coherence on feature level instead. It improves the per-frame features by aggregation of nearby features along the motion paths, and thus improves the video recognition accuracy. Our method significantly improves upon strong singleframe baselines in ImageNet VID [33], especially for more challenging fast moving objects. Our framework is principled, and on par with the best engineered systems winning the ImageNet VID challenges 2016, without additional bells-and-whistles. The code would be released.

Authors

Lu Yuan

7 papers

Xizhou Zhu

7 papers

Jifeng Dai

14 papers

TL;DR

Abstract

Authors

References51 items

Deformable Convolutional Networks

Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks

YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

AdaScan: Adaptive Scan Pooling in Deep Convolutional Neural Networks for Human Action Recognition in Videos

Deep Feature Flow for Video Recognition

On The Stability of Video Detection and Tracking

Convolutional gated recurrent networks for video segmentation

Optical Flow Estimation Using a Spatial Pyramid Network

STFCN: Spatio-Temporal FCN for Semantic Video Segmentation

VideoLSTM convolves, attends and flows for action recognition

R-FCN: Object Detection via Region-based Fully Convolutional Networks

Object Detection from Video Tubelets with Convolutional Neural Networks

T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Seq-NMS for Video Object Detection

Deep Residual Learning for Image Recognition

SSD: Single Shot MultiBox Detector

Visual Tracking with Fully Convolutional Networks

Deep End2End Voxel2Voxel Prediction

Delving Deeper into Convolutional Networks for Learning Video Representations

Action Recognition using Visual Attention

Learning Multi-domain Convolutional Neural Networks for Visual Tracking

Human Action Recognition Using Factorized Spatio-Temporal Convolutional Networks

A Neural Attention Model for Abstractive Sentence Summarization

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Fast R-CNN

FlowNet: Learning Optical Flow with Convolutional Networks

Beyond short snippets: Deep networks for video classification

Describing Videos by Exploiting Temporal Structure

EpicFlow: Edge-preserving interpolation of correspondences for optical flow

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

Learning Spatiotemporal Features with 3D Convolutional Networks

Long-term recurrent convolutional networks for visual recognition and description

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Large-Scale Video Classification with Convolutional Neural Networks

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

ImageNet classification with deep convolutional neural networks

High Accuracy Optical Flow Estimation Based on a Theory for Warping

Determining Optical Flow

Multi-class Multi-object Tracking Using Changing Point Detection

Dropout: a simple way to prevent neural networks from overfitting

Author manuscript, published in "IEEE Intenational Conference on Computer Vision (ICCV), Sydney: Australie (2013)" DeepFlow: Large displacement optical flow with deep matching

Efficient object detection from videos. http://image-net.org/challenges/talks

Efﬁcient object detection from videos

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation

Universität Des Saarlandes Fachrichtung 6.1 – Mathematik a Survey on Variational Optic Flow Methods for Small Displacements a Survey on Variational Optic Flow Methods for Small Displacements

AND T

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names