Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition (2020-03-31T00:00:00.000000Z)

TL;DR

A simple method to disentangle multi-scale graph convolutions and a unified spatial-temporal graph convolutional operator named G3D are presented and a powerful feature extractor named MS-G3D is developed based on which the model outperforms previous state-of-the-art methods on three large-scale datasets.

Abstract

Spatial-temporal graphs have been widely used by skeleton-based action recognition algorithms to model human action dynamics. To capture robust movement patterns from these graphs, long-range and multi-scale context aggregation and spatial-temporal dependency modeling are critical aspects of a powerful feature extractor. However, existing methods have limitations in achieving (1) unbiased long-range joint relationship modeling under multi-scale operators and (2) unobstructed cross-spacetime information flow for capturing complex spatial-temporal dependencies. In this work, we present (1) a simple method to disentangle multi-scale graph convolutions and (2) a unified spatial-temporal graph convolutional operator named G3D. The proposed multi-scale aggregation scheme disentangles the importance of nodes in different neighborhoods for effective long-range modeling. The proposed G3D module leverages dense cross-spacetime edges as skip connections for direct information propagation across the spatial-temporal graph. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets: NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400.

Authors

Wanli Ouyang

26 papers

Hongwen Zhang

4 papers

Ken Ziyu Liu

1 papers

TL;DR

Abstract

Authors

References54 items

Spatio-Temporal Graph Routing for Skeleton-Based Action Recognition

Break the Ceiling: Stronger Multi-scale Deep Graph Convolutional Networks

Skeleton-Based Action Recognition With Directed Graph Neural Networks

Graph WaveNet for Deep Spatial-Temporal Graph Modeling

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding

Graph U-Nets

MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing

Actional-Structural Graph Convolutional Networks for Skeleton-Based Action Recognition

An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition

Simplifying Graph Convolutional Networks

LanczosNet: Multi-Scale Deep Graph Convolutional Networks

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

How Powerful are Graph Neural Networks?

Deep Graph Infomax

Hierarchical Graph Representation Learning with Differentiable Pooling

Videos as Space-Time Region Graphs

Recognizing Human Actions as the Evolution of Pose Estimation Maps

Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition

Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning

Memory Attention Networks for Skeleton-Based Action Recognition

Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation

Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN

Learning Clip Representations for Skeleton-Based 3D Action Recognition

Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition

Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

Adaptive Graph Convolutional Neural Networks

A Closer Look at Spatiotemporal Convolutions for Action Recognition

Graph Attention Networks

Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks

Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting

Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

Inductive Representation Learning on Large Graphs

The Kinetics Human Action Video Dataset

View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data

Semi-Supervised Classification with Graph Convolutional Networks

Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Deep Residual Learning for Image Recognition

Multi-Scale Context Aggregation by Dilated Convolutions

Diffusion-Convolutional Neural Networks

Deep Convolutional Networks on Graph-Structured Data

Hierarchical recurrent neural network for skeleton based action recognition

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Learning Spatiotemporal Features with 3D Convolutional Networks

Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group

Spectral Networks and Locally Connected Networks on Graphs

Wavelets on Graphs via Spectral Graph Theory

A Comprehensive Survey on Graph Neural Networks

Mining actionlet ensemble for action recognition with depth cameras

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names