TASED-Net: Temporally-Aggregating Spatial Encoder-Decoder Network for Video Saliency Detection (2019-08-15T00:00:00.000000Z)

TL;DR

TASED-Net significantly outperforms previous state-of-the-art approaches on all three major large-scale datasets of video saliency detection: DHF1K, Hollywood2, and UCFSports and is especially better at attending to salient moving objects.

Abstract

TASED-Net is a 3D fully-convolutional network architecture for video saliency detection. It consists of two building blocks: first, the encoder network extracts low-resolution spatiotemporal features from an input clip of several consecutive frames, and then the following prediction network decodes the encoded features spatially while aggregating all the temporal information. As a result, a single prediction map is produced from an input clip of multiple frames. Frame-wise saliency maps can be predicted by applying TASED-Net in a sliding-window fashion to a video. The proposed approach assumes that the saliency map of any frame can be predicted by considering a limited number of past frames. The results of our extensive experiments on video saliency detection validate this assumption and demonstrate that our fully-convolutional model with temporal aggregation method is effective. TASED-Net significantly outperforms previous state-of-the-art approaches on all three major large-scale datasets of video saliency detection: DHF1K, Hollywood2, and UCFSports. After analyzing the results qualitatively, we observe that our model is especially better at attending to salient moving objects.

Authors

Jason J. Corso

9 papers

Kyle Min

2 papers

TL;DR

Abstract

Authors

References43 items

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Revisiting Video Saliency: A Large-Scale Benchmark and a New Model

Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification

Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?

Non-local Neural Networks

Predicting Video Saliency with Object-to-Motion CNN and Two-layer Convolutional LSTM

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

The Kinetics Human Action Video Dataset

Deep Visual Attention Prediction

SalGAN: Visual Saliency Prediction with Generative Adversarial Networks

Spatio-Temporal Saliency Networks for Dynamic Saliency Prediction

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

What Do Different Evaluation Metrics Tell Us About Saliency Models?

Recurrent Mixture Density Network for Spatiotemporal Visual Attention

Shallow and Deep Convolutional Networks for Saliency Prediction

SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

You Only Look Once: Unified, Real-Time Object Detection

SALICON: Saliency in Context

U-Net: Convolutional Networks for Biomedical Image Segmentation

Learning Deconvolution Network for Semantic Segmentation

FlowNet: Learning Optical Flow with Convolutional Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Learning Spatiotemporal Features with 3D Convolutional Networks

Fully convolutional networks for semantic segmentation

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition

Static saliency vs. dynamic saliency: a comparative study

Adaptive deconvolutional networks for mid and high level feature learning

A Spatiotemporal Saliency Model for Video Surveillance

Predictive Saliency Maps for Surveillance Videos

Rectified Linear Units Improve Restricted Boltzmann Machines

Actions in context

ImageNet: A large-scale hierarchical image database

Action MACH a spatio-temporal Maximum Average Correlation Height filter for action recognition

Graph-Based Visual Saliency

Long Short-Term Memory

Perceptual integration of motion and form information: is the movement filter involved in form discrimination?

Action Recognition in Realistic Sports Videos

Saliency-Aware Video Compression

A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression

Neural mechanisms of selective visual attention.

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names