Evidential Deep Learning for Open Set Action Recognition (2021-07-21T00:00:00.000000Z)

TL;DR

A Deep Evidential Action Recognition (DEAR) method to recognize actions in an open testing set and a plug-and-play module to debias the learned representation through contrastive learning to mitigate the static bias of video representation.

Abstract

In a real-world scenario, human actions are typically out of the distribution from training data, which requires a model to both recognize the known actions and reject the unknown. Different from image data, video actions are more challenging to be recognized in an open-set setting due to the uncertain temporal dynamics and static bias of human actions. In this paper, we propose a Deep Evidential Action Recognition (DEAR) method to recognize actions in an open testing set. Specifically, we formulate the action recognition problem from the evidential deep learning (EDL) perspective and propose a novel model calibration method to regularize the EDL training. Besides, to mitigate the static bias of video representation, we propose a plug-and-play module to debias the learned representation through contrastive learning. Experimental results show that our DEAR method achieves consistent performance gain on multiple mainstream action recognition models and benchmarks. Code and pre-trained models are available at https://www.rit.edu/actionlab/dear.

Authors

Yu Kong

4 papers

Wentao Bao

5 papers

Qi Yu

3 papers

TL;DR

Abstract

Authors

References67 items

Improving model calibration with accuracy versus uncertainty optimization

Open Set Driver Activity Recognition

Learning Open Set Network with Discriminative Reciprocal Points

Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational Learning

Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts

Temporal Pyramid Network for Action Recognition

Uncertainty-Aware Deep Classifiers Using Generative Models

Conditional Gaussian Distribution Learning for Open Set Recognition

Calibrating Deep Neural Networks using Focal Loss

Mimetics: Towards Understanding Human Actions Out of Context

Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition

Multi-Moments in Time: Learning and Interpreting Models for Multi-Action Video Understanding

Quantifying Classification Uncertainty using Regularized Evidential Neural Networks

Learning De-biased Representations with Biased Representations

Deep Evidential Regression

Open Set Domain Adaptation for Image and Action Recognition

Specifying Weight Priors in Bayesian Deep Neural Networks with Empirical Bayes

Uncertainty Estimation in One-Stage Object Detection

Out-Of-Distribution Detection for Generalized Zero-Shot Action Recognition

C2AE: Class Conditioned Auto-Encoder for Open-Set Recognition

A Comprehensive Survey of Vision-Based Human Action Recognition Methods

Open-set human activity recognition based on micro-Doppler signatures

Learning Not to Learn: Training Deep Neural Networks With Biased Data

SlowFast Networks for Video Recognition

Evaluating Bayesian Deep Learning Methods for Semantic Segmentation

Uncertainty-Aware Audiovisual Activity Recognition Using Deep Bayesian Variational Inference

Recent Advances in Open Set Recognition: A Survey

TSM: Temporal Shift Module for Efficient Video Understanding

BAR: Bayesian Activity Recognition using variational inference

Open-world Learning and Application to Product Classification

RESOUND: Towards Action Recognition Without Representation Bias

Open Set Learning with Counterfactual Images

ODN: Opening the Deep Network for Open-Set Action Recognition

Human Action Recognition and Prediction: A Survey

Evidential Deep Learning to Quantify Classification Uncertainty

Predictive Uncertainty Estimation via Prior Networks

Generative OpenMax for Multi-Class Open Set Classification

On Calibration of Modern Neural Networks

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

Recent advances in video-based human action recognition using deep learning: A review

SphereFace: Deep Hypersphere Embedding for Face Recognition

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

Subjective Logic

Towards Open Set Deep Networks

Obtaining Well Calibrated Probabilities Using Bayesian Binning

Towards Open World Recognition

Multi-class Open Set Recognition Using Probability of Inclusion

Probability Models for Open Set Recognition

Toward Open Set Recognition

UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

Bayesian Active Learning for Classification and Preference Learning

HMDB: A large video database for human motion recognition

Open set face recognition using transduction

Measuring Statistical Dependence with Hilbert-Schmidt Norms

IEEE Transactions on Pattern Analysis and Machine Intelligence

Combination of Evidence in Dempster-Shafer Theory

Multifaceted Uncertainty Estimation for Label-Efficient Deep Learning

Practical uncertainty estimation and out-of-distribution robustness in deep learning. Technical report,

Open- GAN: Open set generative adversarial networks

Longbing Cao, and Anton van den Hengel. Deep learning for anomaly detection: A review

Uncertainty in Deep Learning

Feature Selection via Dependence Maximization

Convolutional Deep Belief Networks on CIFAR-10

Deep learning for anomaly detection : A review . arXiv preprint arXiv :

The mnist database of handwritten digits

Openmmlab's next generation video understanding toolbox and benchmark

MMAction2 Contributors

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names