E2TAD: An Energy-Efficient Tracking-based Action Detector

Published in

arXiv.org(2022)

External Links:

Generate Graph DownloadPDF

TL;DR

This paper presents a tracking-based solution to accurately and efficiently localize predefined key actions spatially (by predicting the associated target IDs and locations) and temporally (by predicting the time in exact frame indices).

Abstract

Video action detection (spatio-temporal action localization) is usually the starting point for human-centric intelligent analysis of videos nowadays. It has high practical impacts for many applications across robotics, security, healthcare, etc. The two-stage paradigm of Faster R-CNN inspires a standard paradigm of video action detection in object detection, i.e., firstly generating person proposals and then classifying their actions. However, none of the existing solutions could provide fine-grained action detection to the"who-when-where-what"level. This paper presents a tracking-based solution to accurately and efficiently localize predefined key actions spatially (by predicting the associated target IDs and locations) and temporally (by predicting the time in exact frame indices). This solution won first place in the UAV-Video Track of 2021 Low-Power Computer Vision Challenge (LPCVC).

Authors

Zhangyang Wang

17 papers

G. Hua

6 papers

Zhenyu Wu

3 papers

References89 items

ByteTrack: Multi-Object Tracking by Associating Every Detection Box

VisDrone-DET2021: The Vision Meets Drone Object detection Challenge Results

YOLOX: Exceeding YOLO Series in 2021

The 2020 Low-Power Computer Vision Challenge

Region-aware Adaptive Instance Normalization for Image Harmonization

E2TAD: An Energy-Efficient Tracking-based Action Detector

Published in

arXiv.org(2022)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Zhangyang Wang

17 papers

G. Hua

6 papers

Zhenyu Wu

3 papers

References89 items

ByteTrack: Multi-Object Tracking by Associating Every Detection Box

VisDrone-DET2021: The Vision Meets Drone Object detection Challenge Results

YOLOX: Exceeding YOLO Series in 2021

The 2020 Low-Power Computer Vision Challenge

Region-aware Adaptive Instance Normalization for Image Harmonization

Haoyuan Miao

1 papers

Pengcheng Pi

1 papers

E2VTS: Energy-Efficient Video Text Spotting from Unmanned Aerial Vehicles

Adaptive Focus for Efficient Video Recognition

Beyond Short Clips: End-to-End Video-Level Learning with Collaborative Memories

2D or not 2D? Adaptive 3D Convolution Selection for Efficient Video Recognition

Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification

PCAL: A Privacy-preserving Intelligent Credit Risk Modeling Framework Based on Adversarial Learning

Privacy-Preserving Deep Action Recognition: An Adversarial Learning Framework and A New Dataset

Time-Based Roofline for Deep Learning Performance Analysis

Context-Aware RCNN: A Baseline for Action Detection in Videos

Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization

YOLOv4: Optimal Speed and Accuracy of Object Detection

Asynchronous Interaction Aggregation for Action Detection

FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking

Tracking Objects as Points

PANDA: A Gigapixel-Level Human-Centric Video Dataset

In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation

STAGE: Spatio-Temporal Attention on Graph Entities for Video Action Detection

Dynamic Convolution: Attention Over Convolution Kernels

Calibrated Domain-Invariant Learning for Highly Generalizable Large Scale Re-Identification

EfficientDet: Scalable and Efficient Object Detection

Towards Real-Time Multi-Object Tracking

Delving Into Robust Object Detection From Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach

Deep learning in video multi-object tracking: A survey

AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates

Importance Estimation for Neural Network Pruning

TIE: Energy-efficient Tensor Train-based Inference Engine for Deep Neural Network

STEP: Spatio-Temporal Progressive Learning for Video Action Detection

Objects as Points

NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

CondConv: Conditionally Parameterized Convolutions for Efficient Inference

FCOS: Fully Convolutional One-Stage Object Detection

ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Methods of Multipliers

Long-Term Feature Banks for Detailed Video Understanding

SlowFast Networks for Video Recognition

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration

Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study

Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices

UCNN: Exploiting Computational Reuse in Deep Neural Networks via Weight Repetition

A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers

NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications

YOLOv3: An Incremental Improvement

Path Aggregation Network for Instance Segmentation

AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

Beyond Part Models: Person Retrieval with Refined Part Pooling

Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification

A 21mW low-power recurrent neural network accelerator with quantization tables for embedded deep learning applications

To prune, or not to prune: exploring the efficacy of pruning for model compression

Learning Efficient Convolutional Networks through Network Slimming

Focal Loss for Dense Object Detection

Deeply-Learned Part-Aligned Representations for Person Re-identification

Deep Representation Learning With Part Loss for Person Re-Identification

Beyond Triplet Loss: A Deep Quadruplet Network for Person Re-identification

Dynamic Computational Time for Visual Attention

Multi-Scale Dense Networks for Resource Efficient Image Classification

In Defense of the Triplet Loss for Person Re-Identification

Simple online and realtime tracking with a deep association metric

Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

YOLO9000: Better, Faster, Stronger

Feature Pyramid Networks for Object Detection

Pruning Filters for Efficient ConvNets

A Siamese Long Short-Term Memory Architecture for Human Re-identification

Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures

DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients

R-FCN: Object Detection via Region-based Fully Convolutional Networks

Deep Attributes Driven Multi-Camera Person Re-identification

Person Re-identification in the Wild

Simple online and realtime tracking

SSD: Single Shot MultiBox Detector

Scalable Person Re-identification: A Benchmark

You Only Look Once: Unified, Real-Time Object Detection

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

DeepReID: Deep Filter Pairing Neural Network for Person Re-identification

Microsoft COCO: Common Objects in Context

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

A new algorithm for image noise reduction using mathematical morphology

Rich feature hierarchies for accurate object detection and semantic segmentation

Runtime Neural Pruning

Ran El-Yaniv, and Yoshua Bengio

Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

arXiv.org

Type

URL

https://arxiv.org

Alternate Names

ArXiv

TL;DR

Abstract

Authors

References89 items

ByteTrack: Multi-Object Tracking by Associating Every Detection Box

VisDrone-DET2021: The Vision Meets Drone Object detection Challenge Results

YOLOX: Exceeding YOLO Series in 2021

The 2020 Low-Power Computer Vision Challenge

Region-aware Adaptive Instance Normalization for Image Harmonization

TL;DR

Abstract

Authors

References89 items

ByteTrack: Multi-Object Tracking by Associating Every Detection Box

VisDrone-DET2021: The Vision Meets Drone Object detection Challenge Results

YOLOX: Exceeding YOLO Series in 2021

The 2020 Low-Power Computer Vision Challenge

Region-aware Adaptive Instance Normalization for Image Harmonization

E2VTS: Energy-Efficient Video Text Spotting from Unmanned Aerial Vehicles

Adaptive Focus for Efficient Video Recognition

Beyond Short Clips: End-to-End Video-Level Learning with Collaborative Memories

2D or not 2D? Adaptive 3D Convolution Selection for Efficient Video Recognition

Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification

PCAL: A Privacy-preserving Intelligent Credit Risk Modeling Framework Based on Adversarial Learning

Privacy-Preserving Deep Action Recognition: An Adversarial Learning Framework and A New Dataset

Time-Based Roofline for Deep Learning Performance Analysis

Context-Aware RCNN: A Baseline for Action Detection in Videos

Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization

YOLOv4: Optimal Speed and Accuracy of Object Detection

Asynchronous Interaction Aggregation for Action Detection

FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking

Tracking Objects as Points

PANDA: A Gigapixel-Level Human-Centric Video Dataset

In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation

STAGE: Spatio-Temporal Attention on Graph Entities for Video Action Detection

Dynamic Convolution: Attention Over Convolution Kernels

Calibrated Domain-Invariant Learning for Highly Generalizable Large Scale Re-Identification

EfficientDet: Scalable and Efficient Object Detection

Towards Real-Time Multi-Object Tracking

Delving Into Robust Object Detection From Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach

Deep learning in video multi-object tracking: A survey

AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates

Importance Estimation for Neural Network Pruning

TIE: Energy-efficient Tensor Train-based Inference Engine for Deep Neural Network

STEP: Spatio-Temporal Progressive Learning for Video Action Detection

Objects as Points

NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

CondConv: Conditionally Parameterized Convolutions for Efficient Inference

FCOS: Fully Convolutional One-Stage Object Detection

ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Methods of Multipliers

Long-Term Feature Banks for Detailed Video Understanding

SlowFast Networks for Video Recognition

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration

Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study

Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices

UCNN: Exploiting Computational Reuse in Deep Neural Networks via Weight Repetition

A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers

NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications

YOLOv3: An Incremental Improvement

Path Aggregation Network for Instance Segmentation

AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

Beyond Part Models: Person Retrieval with Refined Part Pooling

Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification

A 21mW low-power recurrent neural network accelerator with quantization tables for embedded deep learning applications

To prune, or not to prune: exploring the efficacy of pruning for model compression

Learning Efficient Convolutional Networks through Network Slimming

Focal Loss for Dense Object Detection

Deeply-Learned Part-Aligned Representations for Person Re-identification

Deep Representation Learning With Part Loss for Person Re-Identification

Beyond Triplet Loss: A Deep Quadruplet Network for Person Re-identification

Dynamic Computational Time for Visual Attention

Multi-Scale Dense Networks for Resource Efficient Image Classification

In Defense of the Triplet Loss for Person Re-Identification

Simple online and realtime tracking with a deep association metric

Mask R-CNN

Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

YOLO9000: Better, Faster, Stronger