Multi-Person Pose Estimation With Enhanced Channel-Wise and Spatial Information

Su, Kuo-Lan; Yu, Dongdong; Xu, Zexiang; Geng, Xin; Wang, Changhu

doi:10.1109/cvpr.2019.00582

Cited by 156 publications

(72 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nowadays, researchers have made painstaking efforts [17,29,38,40] to accelerate its progress. For examples, CASNet [38] improves the feature representation via adopting the spatial and channel-wise attention. HRNet [40] builds a new strong baseline via elaborated network design.…”

Section: Related Workmentioning

confidence: 99%

“…Multi-person pose estimation devotes to locate body parts for multiple persons in 2D image, such as keypoints on the arms, torsos, and the face [38]. It's fundamental to deal with other high-level tasks, such as human action recognition [6] and human-computer interaction [32].…”

Section: Introductionmentioning

confidence: 99%

“…It's fundamental to deal with other high-level tasks, such as human action recognition [6] and human-computer interaction [32]. Recently, due to the progress of convolution neural network (CNN) [14], most existing methods [4,21,38,49] have achieved remarkable advances in multi-person pose estimation. However, existing supervised methods cannot generalize well to a novel domain without labels or with sparse labels, especially when the new domain has a different distribution.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Alleviating Human-level Shift

Xu¹,

Zou²,

Lin³

2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Human pose estimation has been widely studied with much focus on supervised learning requiring sufficient annotations. However, in real applications, a pretrained pose estimation model usually need be adapted to a novel domain with no labels or sparse labels. Such domain adaptation for 2D pose estimation hasn't been explored. The main reason is that a pose, by nature, has typical topological structure and needs fine-grained features in local keypoints. While existing adaptation methods do not consider topological structure of object-of-interest and they align the whole images coarsely. Therefore, we propose a novel domain adaptation method for multi-person pose estimation to conduct the human-level topological structure alignment and fine-grained feature alignment. Our method consists of three modules: Cross-Attentive Feature Alignment (CAFA), Intra-domain Structure Adaptation (ISA) and Inter-domain Human-Topology Alignment (IHTA) module. The CAFA adopts a bidirectional spatial attention module (BSAM) that focuses on fine-grained local feature correlation between two humans to adaptively aggregate consistent features for adaptation. We adopt ISA only in semi-supervised domain adaptation (SSDA) to exploit the corresponding keypoint semantic relationship for reducing the intra-domain bias. Most importantly, we propose an IHTA to learn more domain-invariant human topological representation for reducing the inter-domain discrepancy. We model the human topological structure via the graph convolution network (GCN), by passing messages on which, high-order relations can be considered. This structure preserving alignment based on GCN is beneficial to the occluded or extreme pose inference. Extensive experiments are conducted on two popular benchmarks and results demonstrate the competency of our method compared with existing supervised approaches. CCS CONCEPTS • Computing methodologies → Activity recognition and understanding; Transfer learning;

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Alleviating Human-level Shift

Xu¹,

Zou²,

Lin³

2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…Nevertheless, the process of feature extraction at all scales will generate redundant information. As far as for the attention mechanism, Su et al [6] proposed a spatial, channel-wise attention residual bottleneck (SCARB) aiming to boost the original residual unit with the attention mechanism. However, SCARB makes spatial attention and channel attention separate, causing the feature maps cannot be maintained and transmitted completely.…”

Section: Hsinchuan Lin Jiaying Zhu and Haifeng Hu ✉mentioning

confidence: 99%

OCPAN: multi‐person pose estimation with more attention on residual information

Lin

Zhu

2020

Electron. lett.

View full text Add to dashboard Cite

Redundant information generated in the process of feature fusion and residual information always remain challenging in multi‐person pose estimation. In this Letter, the authors present an optimised cascaded pyramid attention network composed of two novel modules to reduce the redundant information and highlight the residual information for more accurate results. The first module, called channel optimisation module (COM), optimises the channel at different levels. The proposed COM consisting of convolution layers and two pooling layers performs on the feature maps Conv‐2∼5 of different resolution to reduce the redundant information after convolution operation. The second module, called residual attention bottleneck module (RABM), highlights the residual information in the bottleneck. Through adopting the attention mechanism in the bottleneck, RABM performs a significant role in reducing the redundant information. In contrast to the previous work, they attach more attention to residual information, helping to tackle the challenging problem for the ‘hard’ keypoints. By merging COM and RABM into Cascaded Pyramid Network, the authors' method achieves competitive results of 74.6 AP on the COCO keypoint benchmark, outperforming the state‐of‐the‐art results.

show abstract

“…They can be used for recognition of human action [1][2][3], tracking [4,5] and re-identification [6] of human in online surveillance and human-object interaction [7]. Most of the latest algorithms on pose estimation tend to embed a human detector at the beginning of its data processing unit, such as [9][10][11] which ranked top three in COCO key-point challenge 2019. Those human detection-based algorithms are called top-down methods.…”

Section: Introductionmentioning

confidence: 99%

Training Physical and Geometrical Mid-Points for Multi-person Pose Estimation and Human Detection Under Congestion and Low Resolution

et al. 2020

View full text Add to dashboard Cite

This paper introduces the design and evaluation of NeoPose which is developed for multi-person pose estimation and human detection. The design of NeoPose is targeting the issue of human detection under congested situation and with low resolution in the image. Under such situations, we compared the performance of different versions of NeoPose as well as other existing algorithms in a human detection task. Throughout the task, the usefulness of two kinds of mid-point (physical and geometrical mid-points) and a deconvolution structure was discussed. Experiment results indicated that NeoPose which applied geometrical mid-points and deconvolution structure performed the best in terms of both precision and recall in the evaluation.

show abstract

Multi-Person Pose Estimation With Enhanced Channel-Wise and Spatial Information

Cited by 156 publications

References 24 publications

Alleviating Human-level Shift

Alleviating Human-level Shift

OCPAN: multi‐person pose estimation with more attention on residual information

Training Physical and Geometrical Mid-Points for Multi-person Pose Estimation and Human Detection Under Congestion and Low Resolution

Contact Info

Product

Resources

About