A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications (2022-06-02T00:00:00.000000Z)

TL;DR

To facilitate sports analytics, a toolbox using PaddlePaddle is developed, which supports football, basketball, table tennis and figure skating action recognition, and discusses the challenges and unsolved problems in this area.

Abstract

To understand human behaviors, action recognition based on videos is a common approach. Compared with image-based action recognition, videos provide much more information, reducing the ambiguity of actions. In the last decade, many works focus on datasets, novel models and learning approaches have improved video action recognition to a higher level. However, there are challenges and unsolved problems, in particular in sports analytics where data collection and labeling are more sophisticated, requiring people with domain knowledge and even sport professionals to annotate data. In addition, the actions could be extremely fast and it becomes difficult to recognize them. Moreover, in team sports like football and basketball, one action could involve multiple players, and to correctly recognize them, we need to analyze all players, which is relatively complicated. In this paper, we present a survey on video action recognition for sports analytics. We introduce more than ten types of sports, including team sports, such as football, basketball, volleyball, hockey and individual sports, such as figure skating, gymnastics, table tennis, tennis, diving and badminton. Then we compare numerous existing frameworks for sports analysis to present status quo of video action recognition in both team sports and individual sports. Finally, we discuss the challenges and unsolved problems in this area and to facilitate sports analytics, we develop a toolbox using PaddlePaddle, which supports football, basketball, table tennis and figure skating action recognition.

Authors

Haoyi Xiong

5 papers

D. Dou

9 papers

Jiang Bian

3 papers

TL;DR

Abstract

Authors

References370 items

A Review of Sparse Expert Models in Deep Learning

Dive into Big Model Training

Zero-Shot Temporal Action Detection via Vision-Language Prompting

FenceNet: Fine-grained Footwork Recognition in Fencing

GrOD: Deep Learning with Gradients Orthogonal Decomposition for Knowledge Transfer, Distillation, and Adversarial Training

Analysis of Basketball Technical Movements Based on Human-Computer Interaction with Deep Learning

MM-SEAL: A Large-scale Video Dataset of Multi-person Multi-grained Spatio-temporally Action Localization

Video-based Table Tennis Tracking and Trajectory Prediction Using Spatial-Temporal CNNs based on Deep Learning

BaGuaLu: targeting brain scale pretrained models with over 37 million cores

A Comparative Survey of Deep Active Learning

On Distinctive Image Captioning via Comparing and Reweighting

TFCNet: Temporal Fully Connected Networks for Static Unbiased Temporal Reasoning

Compressing CNN Kernels for Videos Using Tucker Decompositions: Towards Lightweight CNN Applications

Masked Feature Prediction for Self-Supervised Visual Pre-Training

Sports Video: Fine-Grained Action Detection and Classification of Table Tennis Strokes from videos for MediaEval 2021

Two Stream Network for Stroke Detection in Table Tennis

BEVT: BERT Pretraining of Video Transformers

ShuttleNet: Position-aware Fusion of Rally Progress and Player Styles for Stroke Forecasting in Badminton

Ice hockey player identification via transformers and weakly supervised learning

Toward the Perfect Stroke: A Multimodal Approach for Table Tennis Stroke Evaluation

Direct Multi-view Multi-person 3D Pose Estimation

Object-Region Video Transformers

AI Table Tennis: Methods and Challenges

Deep Long-Tailed Learning: A Survey

Player Tracking and Identification in Ice Hockey

Contrast and Order Representations for Video Self-supervised Learning

Three-Stream 3D/1D CNN for Fine-Grained Action Classification and Segmentation in Table Tennis

A Lightweight Fine-Grained Action Recognition Network for Basketball Foul Detection

Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition

Spatio-Temporal Dynamic Inference Network for Group Activity Recognition

Pose is all you need: the pose only group activity recognition system (POGARS)

A Comparative Survey: Benchmarking for Pool-based Active Learning

Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Multiple-criteria Based Active Learning with Fixed-size Determinantal Point Processes

Video Swin Transformer

Recognition of Basketball Player's Shooting Action Based on the Convolutional Neural Network

VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning

BEiT: BERT Pre-Training of Image Transformers

Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers

Puck localization and multi-task event recognition in broadcast hockey videos

Temporal Segmentation of Fine-gained Semantic Action: A Motion-Centered Figure Skating Dataset

MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions

NPU RGBD Dataset and a Feature-Enhanced LSTM-DGCN Method for Action Recognition of Basketball Players+

VideoLT: Large-scale Long-tailed Video Recognition

Emerging Properties in Self-Supervised Vision Transformers

Revisiting Skeleton-based Action Recognition

SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos

Multiscale Vision Transformers

Table Tennis Stroke Recognition Using Two-Dimensional Human Pose Estimation

MGSampler: An Explainable Sampling Strategy for Video Action Recognition

Temporal Query Networks for Fine-grained Video Understanding

Contrastive Learning for Sports Video: Unsupervised Player Classification

Spatiotemporal based table tennis stroke-type assessment

ViViT: A Video Vision Transformer

MoViNets: Mobile Video Networks for Efficient Video Recognition

Shot detection using skeleton position in badminton videos

Open-book Video Captioning with Retrieve-Copy-Generate Network

Optimization of volleyball motion estimation algorithm based on machine vision and wearable devices

Zero-Shot Text-to-Image Generation

Win-Fail Action Recognition

Is Space-Time Attention All You Need for Video Understanding?

Video Transformer Network

Basketball action recognition based on FPGA and particle image

Attention-based LSTM Network for Action Recognition in Sports

Moving the Lab into the Mountains: A Pilot Study of Human Activity Recognition in Unstructured Environments

Camera-based Basketball Scoring Detection Using Convolutional Neural Network

Human Action Recognition From Various Data Modalities: A Review

A Comprehensive Study of Deep Video Action Recognition

A New Action Recognition Framework for Video Highlights Summarization in Sporting Events

SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos

3D attention mechanism for fine-grained classification of table tennis strokes using a Twin Spatio-Temporal Convolutional Neural Networks

Skeleton-based Relational Reasoning for Group Activity Analysis

Learning Three Dimensional Tennis Shots Using Graph Convolutional Networks

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition