Multi-context Attention for Human Pose Estimation

Abstract

In this paper, we propose to incorporate convolutional neural networks with a multi-context attention mechanism into an end-to-end framework for human pose estimation. We adopt stacked hourglass networks to generate attention maps from features at multiple resolutions with various semantics. The Conditional Random Field (CRF) is utilized to model the correlations among neighboring regions in the attention map. We further combine the holistic attention model, which focuses on the global consistency of the full human body, and the body part attention model, which focuses on the detailed description for different body parts. Hence our model has the ability to focus on different granularity from local salient regions to global semanticconsistent spaces. Additionally, we design novel Hourglass Residual Units (HRUs) to increase the receptive field of the network. These units are extensions of residual units with a side branch incorporating filters with larger receptive fields, hence features with various scales are learned and combined within the HRUs. The effectiveness of the proposed multi-context attention mechanism and the hourglass residual units is evaluated on two widely used human pose estimation benchmarks. Our approach outperforms all existing methods on both benchmarks over all the body parts.

Learning Context Flexible Attention Model for Long-Term Visual Place Recognition

“…Inspired by recent studies which show that features extracted from different layers of CNNs capture different semantic structures [11][12][13], in this paper we also estimate the attention map using multi-semantic contexts. Features from lower convolutional layers generally respond to low level image features, such as corners or edges, while those from higher layers focus on structures that are more semantically meaningful, such as human faces or buildings.…”

“…In particular, the context information, which is generally addressed by selecting image areas surrounding the localized, target image regions [11], have been shown to provide useful cues to estimate the attention. However, the contextual areas around each target region are usually defined manually in the shape of rectangular bounding boxes [6,7,11,27]. Such manually defined focus areas often may be suboptimal, since the actualy contextual area can vary under different situations.…”

“…It has been shown that features extracted from lower layers most often correspond to low-level image features, such as edges or corners, while activations from higher layers respond to more semantically meaningful shapes, such as trees or faces [13]. Inspired by this success of multi-scale features [6,11,31], here we propose to learn attention by fusing context extracted from multiple layers of the network.…”

“…When calculating the relevance , we construct the correlation matrix: Ω ∈ × , which is a trainable matrix that can be automatically adapted to relate and . Compared to many other existing attention learning approaches, which manually define the contextual region as rectangular bounding boxes [6,7,11,27] (denoted in red in Figure 2), our approach adopts to adaptively learn the contextual area for each spatial region. In theory, this can define an adaptive contextual area of any shape.…”

Multi-Level Network for High-Speed Multi-Person Pose Estimation

“…Only relying on detection with local information cannot distinguish the joint type from the left or right. To handle this problem, some previous methods [8][9] [10] were proposed by increasing the receptive fields to capture global context information to support the discrimination of left/right joint types. In this way, the discrimination of one joint may use the context information of multiple neighbouring body parts.…”