DeepSportLab: a Unified Framework for Ball Detection, Player Instance Segmentation and Pose Estimation in Team Sports Scenes (2021-12-01T00:00:00.000000Z)

TL;DR

A single model is proposed that simultaneously predicts the ball and the player mask and pose by combining the part intensity fields and the spatial embeddings principles, achieving comparable performance to the SoA models addressing each individual task separately.

Abstract

This paper presents a unified framework to (i) locate the ball, (ii) predict the pose, and (iii) segment the instance mask of players in team sports scenes. Those problems are of high interest in automated sports analytics, production, and broadcast. A common practice is to individually solve each problem by exploiting universal state-of-the-art models, \eg, Panoptic-DeepLab for player segmentation. In addition to the increased complexity resulting from the multiplication of single-task models, the use of the off-the-shelf models also impedes the performance due to the complexity and specificity of the team sports scenes, such as strong occlusion and motion blur. To circumvent those limitations, our paper proposes to train a single model that simultaneously predicts the ball and the player mask and pose by combining the part intensity fields and the spatial embeddings principles. Part intensity fields provide the ball and player location, as well as player joints location. Spatial embeddings are then exploited to associate player instance pixels to their respective player center, but also to group player joints into skeletons. We demonstrate the effectiveness of the proposed model on the DeepSport basketball dataset, achieving comparable performance to the SoA models addressing each individual task separately.

Authors

C. Vleeschouwer

11 papers

Gabriel Van Zandycke

4 papers

M. Istasse

2 papers

TL;DR

Abstract

Authors

References47 items

OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association

Multimodal and multiview distillation for real-time player detection on a football field

Real-time CNN-based Segmentation Architecture for Ball Detection in a Single View Setup

Frame-Level Event Detection in Athletics Videos with Pose-Based Convolutional Sequence Networks

Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation

Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth

Refining Joint Locations for Human Pose Tracking in Sports Videos

Associative Embedding for Team Discrimination

PifPaf: Composite Fields for Human Pose Estimation

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Mask R-CNN and Optical Flow Based Method for Detection and Marking of Handball Actions

Evaluation of Image Representations for Player Detection in Field Sports Using Convolutional Neural Networks

Semi-convolutional Operators for Instance Segmentation

MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network

Lightweight convolutional neural networks for player detection and classification

Box2Pix: Single-Shot Instance Segmentation by Assigning Pixels to Object Boxes

A Bottom-Up Approach Based on Semantics for the Interpretation of the Main Camera Stream in Soccer Games

Convolutional Neural Networks Based Ball Detection in Tennis Games

Kinematic Pose Rectification for Performance Analysis and Retrieval in Sports

PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model

Panoptic Segmentation

Recurrent Pixel Embedding for Instance Grouping

Semantic Instance Segmentation with a Discriminative Loss Function

Benchmarking and Error Diagnosis in Multi-instance Pose Estimation

Athlete Pose Estimation by a Global-Local Network

Soccer player recognition using spatial constellation features and jersey number recognition

Continuous Video to Simple Signals for Swimming Stroke Detection with Convolutional Neural Networks

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

ICNet for Real-Time Semantic Segmentation on High-Resolution Images

Semantic Instance Segmentation via Deep Metric Learning

Mask R-CNN

Towards Accurate Multi-person Pose Estimation in the Wild

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

Associative Embedding: End-to-End Learning for Joint Detection and Grouping

Ball Localization for Robocup Soccer Using Convolutional Neural Networks

Deep Residual Learning for Image Recognition

Multi-Scale Context Aggregation by Dilated Convolutions

Proposal-Free Network for Instance-Level Object Segmentation

Fast R-CNN

Microsoft COCO: Common Objects in Context

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Athlete Pose Estimation from Monocular TV Sports Footage

Formulating Team-Sport Video Summarization as a Resource Allocation Problem

Automatic production of personalized basketball video summaries from multi-sensored data

Browsing Sport Content through an Interactive H.264 Streaming Session

Real-time single image and video super-resolution using an efficient subpixel convolutional neural network

DeepSport data set

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names