Deep High-Resolution Representation Learning for Visual Recognition

Published in

IEEE Transactions on Pattern Analysis and Machi...(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

The superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, is shown, suggesting that the HRNet is a stronger backbone for computer vision problems.

Abstract

High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions in series (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams in parallel and (ii) repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems. All the codes are available at https://github.com/HRNet.

Authors

Ke Sun

6 papers

Bin Xiao

6 papers

Jingdong Wang

10 papers

References197 items

Object Detection

Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation

VisDrone-VID2019: The Vision Meets Drone Object Detection in Video Challenge Results

VisDrone-DET2019: The Vision Meets Drone Object Detection in Image Challenge Results

Object-Contextual Representations for Semantic Segmentation

Deep High-Resolution Representation Learning for Visual Recognition

Published in

IEEE Transactions on Pattern Analysis and Machi...(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Ke Sun

6 papers

Bin Xiao

6 papers

Jingdong Wang

10 papers

References197 items

Object Detection

Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation

VisDrone-VID2019: The Vision Meets Drone Object Detection in Video Challenge Results

VisDrone-DET2019: The Vision Meets Drone Object Detection in Image Challenge Results

Object-Contextual Representations for Semantic Segmentation

Tianheng Cheng

8 papers

Borui Jiang

2 papers

Chaorui Deng

1 papers

Mingkui Tan

7 papers

Xinggang Wang

22 papers

Asymmetric Non-Local Neural Networks for Semantic Segmentation

Interlaced Sparse Self-Attention for Semantic Segmentation

Progressive Image Inpainting with Full-Resolution Residual Network

Cascade R-CNN: High Quality Object Detection and Instance Segmentation

MMDetection: Open MMLab Detection Toolbox and Benchmark

Semantic Correlation Promoted Shape-Variant Context for Segmentation

Co-Occurrent Features in Semantic Segmentation

NTIRE 2019 Challenge on Image Enhancement: Methods and Results

Learning Temporal Pose Estimation from Sparsely-Labeled Videos

Adaptive Pyramid Context Network for Semantic Segmentation

NTIRE 2019 Image Dehazing Challenge Report

High-Resolution Network for Photorealistic Style Transfer

CenterNet: Keypoint Triplets for Object Detection

High-Resolution Representations for Labeling Pixels and Regions

FCOS: Fully Convolutional One-Stage Object Detection

Deep High-Resolution Representation Learning for Human Pose Estimation

Hybrid Task Cascade for Instance Segmentation

Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation

Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment

CCNet: Criss-Cross Attention for Semantic Segmentation

Rethinking ImageNet Pre-Training

M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network

Sequential Context Encoding for Duplicate Removal

Devil in the Details: Towards Accurate Single and Multiple Human Parsing

Dual Attention Network for Scene Segmentation

Context Refinement for Object Detection

Mutual Learning to Adapt for Joint Human Parsing and Pose Estimation

A Deeply-Initialized Coarse-to-fine Ensemble of Regression Trees for Face Alignment

PSANet: Point-wise Spatial Attention Network for Scene Parsing

DetNet: Design Backbone for Object Detection

Parallel Feature Pyramid Network for Object Detection

Deeply Learned Compositional Models for Human Pose Estimation

OCNet: Object Context Network for Scene Parsing

Deep Feature Pyramid Reconfiguration for Object Detection

Quantized Densely Connected U-Nets for Efficient Landmark Localization

CornerNet: Detecting Objects as Paired Keypoints

BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation

Unified Perceptual Parsing for Scene Understanding

Macro-Micro Adversarial Network for Human Parsing

UNet++: A Nested U-Net Architecture for Medical Image Segmentation

MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network

Scale-Transferrable Object Detection

NU-Net: Deep Residual Wide Field of View Convolutional Neural Network for Semantic Segmentation

IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks

Context Contrasted Feature and Gated Multi-scale Aggregation for Scene Segmentation

Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors

Direct Shape Regression Networks for End-to-End Face Alignment

Look at Boundary: A Boundary-Aware Face Alignment Algorithm

Pyramid Attention Network for Semantic Segmentation

PAD-Net: Multi-tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing

SNIPER: Efficient Multi-Scale Training

R-FCN++: Towards Accurate Region-Based Fully Convolutional Networks for Object Detection

Learning a Discriminative Feature Network for Semantic Segmentation

Simple Baselines for Human Pose Estimation and Tracking

ExFuse: Enhancing Feature Fusion for Semantic Segmentation

Look into Person: Joint Body Parsing & Pose Estimation Network and a New Benchmark

Interleaved Structured Sparse Convolutional Neural Networks

Multi-scale Location-Aware Kernel Representation for Object Detection

Multi-Scale Structure-Aware Network for Human Pose Estimation

Adaptive Affinity Fields for Semantic Segmentation

Context Encoding for Semantic Segmentation

PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model

Dynamic-Structured Semantic Propagation Network

Style Aggregated Network for Facial Landmark Detection

Path Aggregation Network for Instance Segmentation

Disentangling 3D Pose in a Dendritic CNN for Unconstrained 2D Face Alignment

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Deep Regionlets for Object Detection

Cascade R-CNN: Delving Into High Quality Object Detection

Single-Shot Object Detection with Enriched Semantics

Relation Networks for Object Detection

An Analysis of Scale Invariance in Object Detection - SNIP

Integral Human Pose Regression

Receptive Field Block Net for Accurate and Fast Object Detection

Cascaded Pyramid Network for Multi-person Pose Estimation

MegDet: A Large Mini-Batch Object Detector

Single-Shot Refinement Neural Network for Object Detection

Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks

AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding

Improving Object Localization with Fitness NMS and Bounded IoU Loss

Interleaved Group Convolutions

A Coarse-Fine Network for Keypoint Localization

Scale-Adaptive Convolutions for Scene Parsing

Joint Multi-View Face Alignment in the Wild

Stacked Deconvolutional Network for Semantic Segmentation

CoupleNet: Coupling Global Structure with Local Parts for Object Detection

Focal Loss for Dense Object Detection

Learning Feature Pyramids for Human Pose Estimation

Residual Conv-Deconv Grid Network for Semantic Segmentation

A Deep Regression Architecture with Two-Stage Re-initialization for High Performance Facial Landmark Detection

Self-Supervised Neural Aggregation Networks for Human Parsing

Stacked Hourglass Network for Robust Facial Landmark Localisation

Gated Feedback Refinement Network for Dense Image Labeling

Leveraging Intra and Inter-Dataset Variations for Robust Face Alignment

100

Rethinking Atrous Convolution for Semantic Image Segmentation

101

Deep Alignment Network: A Convolutional Neural Network for Robust Face Alignment

102

Dilated Residual Networks

103

Recurrent Scene Parsing with Perspective Understanding in the Loop

104

Adversarial PoseNet: A Structure-Aware Convolutional Network for Human Pose Estimation

105

Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade

106

DeNet: Scalable Real-Time Object Detection with Directed Sparse Sampling

107

Multi-Scale Dense Convolutional Networks for Efficient Prediction

108

How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks)

110

SORT: Second-Order Response Transform for Visual Recognition

111

Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing

112

Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network

113

Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources

114

Understanding Convolution for Semantic Segmentation

115

Multi-context Attention for Human Pose Estimation

116

Towards Accurate Multi-person Pose Estimation in the Wild

117

Feature Pyramid Networks for Object Detection

118

Pyramid Scene Parsing Network

119

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

120

RMPE: Regional Multi-person Pose Estimation

121

Video Scene Parsing with Predictive Feature Learning

122

Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

123

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

124

Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes

125

Deep Convolutional Neural Networks with Merge-and-Run Mappings

126

RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation

127

Associative Embedding: End-to-End Learning for Joint Detection and Grouping

128

Dynamic Attention-Controlled Cascaded Shape Regression Exploiting Training Data Augmentation and Fuzzy-Set Sample Weighting

129

Aggregated Residual Transformations for Deep Neural Networks

130

Robust Facial Landmark Detection via Recurrent Attentive-Refinement Networks

131

Human Pose Estimation via Convolutional Part Heatmap Regression

132

A Recurrent Encoder-Decoder Network for Sequential Face Alignment

133

A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection

134

Occlusion-Free Face Alignment: Deep Regression Networks Coupled with De-Corrupt AutoEncoders

135

Unconstrained Face Alignment via Cascaded Compositional Learning

136

Mnemonic Descent Method: A Recurrent Process Applied for End-to-End Face Alignment

137

Convolutional Neural Fabrics

138

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

139

Deeply-Fused Nets

140

DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model

141

Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation

142

The Cityscapes Dataset for Semantic Urban Scene Understanding

143

Human Pose Estimation Using Deep Consensus Voting

144

Stacked Hourglass Networks for Human Pose Estimation

145

Approaching human level facial landmark localization by deep learning

146

M3 CSR: Multi-view, multi-scale and multi-component cascade shape regression

147

Face Alignment via Regressing Local Binary Features

148

Deep Residual Learning for Image Recognition

149

Higher Order Conditional Random Fields in Deep Neural Networks

150

Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation

151

DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation

152

WIDER FACE: A Face Detection Benchmark

153

Attention to Scale: Scale-Aware Semantic Image Segmentation

154

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

155

Interlinked Convolutional Neural Networks for Face Parsing

156

Bottom-Up and Top-Down Reasoning with Hierarchical Rectified Gaussians

157

Cascaded Collaborative Regression for Robust Facial Landmark Detection Trained Using a Mixture of Synthetic and Real Images With Dynamic Weighting

158

Face alignment by coarse-to-fine shape searching

159

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

160

U-Net: Convolutional Networks for Biomedical Image Segmentation

161

Learning Deconvolution Network for Semantic Segmentation

162

Holistically-Nested Edge Detection

163

Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation

164

BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation

165

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

166

Adam: A Method for Stochastic Optimization

167

Locally Scale-Invariant Convolutional Neural Networks

168

Scale-Invariant Convolutional Neural Networks

169

Efficient object localization using Convolutional Networks

170

Fully convolutional networks for semantic segmentation

171

Going deeper with convolutions

172

Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment

173

Very Deep Convolutional Networks for Large-Scale Image Recognition

174

ImageNet Large Scale Visual Recognition Challenge

175

The Role of Context for Object Detection and Semantic Segmentation in the Wild

176

One millisecond face alignment with an ensemble of regression trees

177

Face Alignment at 3000 FPS via Regressing Local Binary Features

178

Occlusion Coherence: Localizing Occluded Faces with a Hierarchical Deformable Part Model

179

Microsoft COCO: Common Objects in Context

180

OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

181

Learn to Combine Multiple Hypotheses for Accurate Face Alignment

182

300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge

183

Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild

184

Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade

185

Robust Face Landmark Estimation under Occlusion

186

Localizing Parts of Faces Using a Consensus of Exemplars

187

Pose-Free Facial Landmark Fitting via Optimized Part Mixtures and Cascaded Deformable Shape Model

188

Supervised Descent Method and Its Applications to Face Alignment

189

ImageNet classification with deep convolutional neural networks

190

Interactive Facial Feature Localization

191

Face Alignment by Explicit Shape Regression

192

Face detection, pose estimation, and landmark localization in the wild

193

Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization

194

Centernet: Authorized licensed use limited to: SOUTH CHINA UNIVERSITY OF TECHNOLOGY

195

Mscoco keypoints challenge 2018

196

The devil is in the decoder

197