Deeply Learned Compositional Models for Human Pose Estimation

Tang, Wei; Yu, Pei; Wu, Ying

doi:10.1007/978-3-030-01219-9_12

Cited by 224 publications

(136 citation statements)

References 43 publications

Supporting

Mentioning

132

Contrasting

Order By: Relevance

“…High-to-low and low-to-high. The high-to-low process aims to generate low-resolution and high-level representations, and the low-to-high process aims to produce highresolution representations [4,11,23,72,40,62]. Both the two processes are possibly repeated several times for boosting the performance [77,40,14].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deep High-Resolution Representation Learning for Human Pose Estimation

Sun

Xiao

Liu

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

4,147

2,538

View full text Add to dashboard Cite

In this paper, we are interested in the human pose estimation problem with a focus on learning reliable highresolution representations. Most existing methods recover high-resolution representations from low-resolution representations produced by a high-to-low resolution network. Instead, our proposed network maintains high-resolution representations through the whole process.We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutliresolution subnetworks in parallel. We conduct repeated multi-scale fusions such that each of the high-to-low resolution representations receives information from other parallel representations over and over, leading to rich highresolution representations. As a result, the predicted keypoint heatmap is potentially more accurate and spatially more precise. We empirically demonstrate the effectiveness of our network through the superior pose estimation results over two benchmark datasets: the COCO keypoint detection dataset and the MPII Human Pose dataset. In addition, we show the superiority of our network in pose tracking on the PoseTrack dataset. The code and models have been publicly available at https://github.com/leoxiaobin/ deep-high-resolution-net.pytorch.

show abstract

Section: Related Workmentioning

confidence: 99%

“…The testing procedure is almost the same to that in COCO except that we adopt the standard testing strategy to use the provided person boxes instead of detected person boxes. Following [14,77,62], a six-scale pyramid testing procedure is performed. Evaluation metric.…”

Section: Mpii Human Pose Estimationmentioning

confidence: 99%

Deep High-Resolution Representation Learning for Human Pose Estimation

Sun

Xiao

Liu

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

4,147

2,538

View full text Add to dashboard Cite

show abstract

“…The spatial model for articulated pose is either based on tree-structured graphical models [7], [8], [9], [10], [11], [12], [13], which parametrically encode the spatial relationship between adjacent parts following a kinematic chain, or non-tree models [14], [15], [16], [17], [18] that augment the tree structure with additional edges to capture occlusion, symmetry, and longrange relationships. To obtain reliable local observations of body parts, Convolutional Neural Networks (CNNs) have been widely used, and have significantly boosted the accuracy on body pose estimation [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32]. Tompson et al [23] used a deep architecture with a graphical model whose parameters are learned jointly with the network.…”

Section: Single Person Pose Estimationmentioning

confidence: 99%

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Cao

Hidalgo

Simon

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

3,471

2,155

View full text Add to dashboard Cite

Realtime multi-person 2D pose estimation is a key component in enabling machines to have an understanding of people in images and videos. In this work, we present a realtime approach to detect the 2D pose of multiple people in an image. The proposed method uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. This bottom-up system achieves high accuracy and realtime performance, regardless of the number of people in the image. In previous work, PAFs and body part location estimation were refined simultaneously across training stages. We demonstrate that a PAF-only refinement rather than both PAF and body part location refinement results in a substantial increase in both runtime performance and accuracy. We also present the first combined body and foot keypoint detector, based on an internal annotated foot dataset that we have publicly released. We show that the combined detector not only reduces the inference time compared to running them sequentially, but also maintains the accuracy of each component individually. This work has culminated in the release of OpenPose, the first open-source realtime system for multi-person 2D pose detection, including body, foot, hand, and facial keypoints.Index Terms-2D human pose estimation, 2D foot keypoint estimation, real-time, multiple person, part affinity fields.

show abstract

“…An upsample process can be used to gradually recover the high-resolution representations from the low-resolution representations. The upsample subnetwork could be a symmetric version of the downsample process (e.g., VGGNet), with skipping connection over some mirrored layers to transform the pooling indices, e.g., SegNet [3] and DeconvNet [85], or copying the feature maps, e.g., U-Net [95] and Hourglass [6], [7], [21], [24], [51], [83], [109], [131], [132], encoder-decoder [90], and so on. An extension of U-Net, full-resolution residual network [92], introduces an extra full-resolution stream that carries information at the full image resolution, to replace the skip connections, and each unit in the downsample and upsample subnetworks receives information from and sends information to the full-resolution stream.…”

Section: Related Workmentioning

confidence: 99%

Deep High-Resolution Representation Learning for Visual Recognition

Wang

Sun

Cheng

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

2,855

1,479

View full text Add to dashboard Cite

High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions in series (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams in parallel; (ii) Repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems. All the codes are available at https://github.com/HRNet. ! 1 INTRODUCTION D EEP convolutional neural networks (DCNNs) have achieved state-of-the-art results in many computer vision tasks, such as image classification, object detection, semantic segmentation, human pose estimation, and so on. The strength is that DCNNs are able to learn richer representations than conventional hand-crafted representations. Most recently-developed classification networks, including AlexNet [59], VGGNet [101], GoogleNet [108], ResNet [39], etc., follow the design rule of LeNet-5 [61]. This is depicted in Figure 1 (a): gradually reduce the spatial size of the feature maps, connect the convolutions from high resolution to low resolution in series, and lead to a low-resolution representation, which is further processed for classification.High-resolution representations are needed for positionsensitive tasks, e.g., semantic segmentation, human pose estimation, and object detection. The previous state-of-the-art methods adopt the high-resolution recovery process to raise the representation resolution from the low-resolution representation outputted by a classification or classification-like network as depicted in Figure 1 (b), e.g., Hourglass [83], Seg-Net [3], DeconvNet [85], U-Net [95], SimpleBaseline [124], and encoder-decoder [90]. In addition, dilated convolutions are used to remove some down-sample layers and thus yield medium-resolution representations [15], [144].We present a novel architecture, namely High-Resolution Net (HRNet), which is able to maintain high-resolution representations through the whole process. We start from a highresolution convolution stream, gradually add high-to-low resolution convolution streams one by one, and connect the multi-resolution streams in parallel. The resulting network • J. Wang is with Microsoft Research,

show abstract

Deeply Learned Compositional Models for Human Pose Estimation

Cited by 224 publications

References 43 publications

Deep High-Resolution Representation Learning for Human Pose Estimation

Deep High-Resolution Representation Learning for Human Pose Estimation

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Deep High-Resolution Representation Learning for Visual Recognition

Contact Info

Product

Resources

About