AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time (2022-11-07T00:00:00.000000Z)

TL;DR

This article presents AlphaPose, a system that can perform accurate whole-body pose estimation and tracking jointly while running in realtime, and proposes several new techniques: Symmetric Integral Keypoint Regression (SIKR) for fast and fine localization, Parametric Pose Non-Maximum-Suppression (P-NMS) for eliminating redundant human detections and Pose Aware Identity Embedding for jointly posing and tracking.

Abstract

Accurate whole-body multi-person pose estimation and tracking is an important yet challenging topic in computer vision. To capture the subtle actions of humans for complex behavior analysis, whole-body pose estimation including the face, body, hand and foot is essential over conventional body-only pose estimation. In this article, we present AlphaPose, a system that can perform accurate whole-body pose estimation and tracking jointly while running in realtime. To this end, we propose several new techniques: Symmetric Integral Keypoint Regression (SIKR) for fast and fine localization, Parametric Pose Non-Maximum-Suppression (P-NMS) for eliminating redundant human detections and Pose Aware Identity Embedding for jointly pose estimation and tracking. During training, we resort to Part-Guided Proposal Generator (PGPG) and multi-domain knowledge distillation to further improve the accuracy. Our method is able to localize whole-body keypoints accurately and tracks humans simultaneously given inaccurate bounding boxes and redundant detections. We show a significant improvement over current state-of-the-art methods in both speed and accuracy on COCO-wholebody, COCO, PoseTrack, and our proposed Halpe-FullBody pose estimation dataset. Our model, source codes and dataset are made publicly available at https://github.com/MVIG-SJTU/AlphaPose.

Authors

Jiefeng Li

3 papers

Haoshu Fang

14 papers

Cewu Lu

4 papers

TL;DR

Abstract

Authors

References90 items

Human Pose Regression with Residual Log-likelihood Estimation

YOLOX: Exceeding YOLO Series in 2021

Learning Dynamics via Graph Neural Networks for Human Pose Estimation and Tracking

OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association

InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

Whole-Body Human Pose Estimation in the Wild

Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation

PandaNet: Anchor-Based Single-Shot Multi-Person 3D Pose Estimation

TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model

Combining Detection and Tracking for Human Pose Estimation in Videos

15 Keypoints Is All You Need

PyTorch: An Imperative Style, High-Performance Deep Learning Library

EfficientDet: Scalable and Efficient Object Detection

DirectPose: Direct End-to-End Multi-Person Pose Estimation

Mask-Pose Cascaded CNN for 2D Hand Pose Estimation From Single Color Image

Single-Network Whole-Body Pose Estimation

Towards Real-Time Multi-Object Tracking

FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB Images

HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation

Single-Stage Multi-Person Pose Machines

FastPose: Towards Real-time Pose Estimation and Tracking via Scale-normalized Multi-task Networks

Learning Temporal Pose Estimation from Sparsely-Labeled Videos

Pose estimator and tracker using temporal flow maps for limbs

LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking

Objects as Points

Multi-Person Articulated Tracking With Spatial and Temporal Embeddings

PifPaf: Composite Fields for Human Pose Estimation

Deep High-Resolution Representation Learning for Human Pose Estimation

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark

Efficient Online Multi-Person 2D Pose Tracking With Recurrent Spatio-Temporal Affinity Fields

Multi-Domain Pose Network for Multi-Person Pose Estimation and Tracking

A Top-Down Approach to Articulated Human Pose Estimation and Tracking

3D Human Pose Estimation With 2D Marginal Heatmaps

Look at Boundary: A Boundary-Aware Face Alignment Algorithm

Human Computer Interaction with Head Pose, Eye Gaze and Body Gestures

Joint Flow: Temporal Flow Fields for Multi Person Tracking

Simple Baselines for Human Pose Estimation and Tracking

Regularisation of neural networks by enforcing Lipschitz continuity

YOLOv3: An Incremental Improvement

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning

Human Action Adverb Recognition: ADHA Dataset and a Three-Stream Hybrid Model

Pose Flow: Efficient Online Pose Tracking

Detect-and-Track: Efficient Pose Estimation in Videos

Integral Human Pose Regression

Cascaded Pyramid Network for Multi-person Pose Estimation

PoseTrack: A Benchmark for Human Pose Estimation and Tracking

Human Pose Regression by Combining Indirect Part Detection and Contextual Information

Large-scale Multiview 3D Hand Pose Dataset

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

Soft-NMS — Improving Object Detection with One Line of Code

On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning

Mask R-CNN

Deformable Convolutional Networks

Understanding Convolution for Semantic Segmentation

Learning to Detect Human-Object Interactions

Towards Accurate Multi-person Pose Estimation in the Wild

RMPE: Regional Multi-person Pose Estimation

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

Associative Embedding: End-to-End Learning for Joint Detection and Grouping

Multi-person Pose Estimation with Local Joint-to-Person Associations

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model

LIFT: Learned Invariant Feature Transform

Stacked Hourglass Networks for Human Pose Estimation

MOT16: A Benchmark for Multi-Object Tracking

300 Faces In-The-Wild Challenge: database and results

Convolutional Pose Machines

Deep Residual Learning for Image Recognition

MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems

DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation

Learning Visual Feature Spaces for Robotic Manipulation with Deep Spatial Autoencoders

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Adam: A Method for Stochastic Optimization

Parsing occluded people by flexible compositions

Using k-Poselets for Detecting People and Localizing Their Keypoints