Towards High Performance Video Object Detection for Mobiles (2018-04-16T00:00:00.000000Z)

TL;DR

A light weight network architecture for video object detection on mobiles, with a very small network for establishing correspondence across frames, and a flow-guided GRU module designed to effectively aggregate features on key frames.

Abstract

Despite the recent success of video object detection on Desktop GPUs, its architecture is still far too heavy for mobiles. It is also unclear whether the key principles of sparse feature propagation and multi-frame feature aggregation apply at very limited computational resources. In this paper, we present a light weight network architecture for video object detection on mobiles. Light weight image object detector is applied on sparse key frames. A very small network, Light Flow, is designed for establishing correspondence across frames. A flow-guided GRU module is designed to effectively aggregate features on key frames. For non-key frames, sparse feature propagation is performed. The whole network can be trained end-to-end. The proposed system achieves 60.2% mAP score at speed of 25.6 fps on mobiles (e.g., HuaWei Mate 8).

Authors

Lu Yuan

7 papers

Xizhou Zhu

7 papers

Jifeng Dai

14 papers

TL;DR

Abstract

Authors

References54 items

Tiny SSD: A Tiny Single-Shot Detection Deep Convolutional Neural Network for Real-Time Embedded Object Detection

Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Towards High Performance Video Object Detection

Light-Head R-CNN: In Defense of Two-Stage Object Detector

Mobile Video Object Detection with Temporally-Aware Feature Maps

Fast YOLO: A Fast You Only Look Once System for Real-time Embedded Object Detection in Video

Focal Loss for Dense Object Detection

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Flow-Guided Feature Aggregation for Video Object Detection

Deformable Convolutional Networks

YOLO9000: Better, Faster, Stronger

Feature Pyramid Networks for Object Detection

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors

Deep Feature Flow for Video Recognition

Aggregated Residual Transformations for Deep Neural Networks

Deconvolution and Checkerboard Artifacts

Xception: Deep Learning with Depthwise Separable Convolutions

Densely Connected Convolutional Networks

VideoLSTM convolves, attends and flows for action recognition

R-FCN: Object Detection via Region-based Fully Convolutional Networks

T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Deep Residual Learning for Image Recognition

SSD: Single Shot MultiBox Detector

Delving Deeper into Convolutional Networks for Learning Video Representations

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

You Only Look Once: Unified, Real-Time Object Detection

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Fast R-CNN

FlowNet: Learning Optical Flow with Convolutional Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Microsoft COCO: Common Objects in Context

Rigid-Motion Scattering for Texture Classification

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Long Short-Term Memory

Finding Structure in Time

Mask R-CNN

Darknet: Open source neural networks in c

- LEVEL ACCURACY WITH 50 X FEWER PARAMETERS AND < 0 . 5 MB MODEL SIZE

Multi-class Multi-object Tracking Using Changing Point Detection

Fully convolutional models for semantic segmentation

Rectifier Nonlinearities Improve Neural Network Acoustic Models

International Journal of Computer Vision manuscript No. (will be inserted by the editor) The PASCAL Visual Object Classes (VOC) Challenge

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names