Part-Aware Context Network for Human Parsing

Zhang, Xiaomei; Chen, Yingying; Zhu, Bingke; Wang, Jinqiao; Tang, Ming

doi:10.1109/cvpr42600.2020.00899

Cited by 56 publications

(22 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fine-grained human semantic segmentation, as one of the central tasks in human understanding, has applications in human-centric vision [41,66,43], human-robot interaction [11] and fashion analysis [46]. However, previous studies mainly focus on category-level human parsing [32,15,12,35,47,63,64,49]; only very few human parsers are specifically designed for the instance-aware setting. As of to date, there exist two paradigms for instance-aware human parsing: top-down and bottom-up.…”

Section: Related Workmentioning

confidence: 99%

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

Zhou

Yang

Liu

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

To address the challenging task of instance-aware human part parsing, a new bottom-up regime is proposed to learn category-level human semantic segmentation as well as multi-person pose estimation in a joint and end-to-end manner. It is a compact, efficient and powerful framework that exploits structural information over different human granularities and eases the difficulty of person partitioning. Specifically, a dense-to-sparse projection field, which allows explicitly associating dense human semantics with sparse keypoints, is learnt and progressively improved over the network feature pyramid for robustness. Then, the difficult pixel grouping problem is cast as an easier, multiperson joint assembling task. By formulating joint association as maximum-weight bipartite matching, a differentiable solution is developed to exploit projected gradient descent and Dykstra's cyclic projection algorithm. This makes our method end-to-end trainable and allows back-propagating the grouping error to directly supervise multi-granularity human representation learning. This is distinguished from current bottom-up human parsers or pose estimators which require sophisticated post-processing or heuristic greedy algorithms. Experiments on three instance-aware human parsing datasets show that our model outperforms other bottom-up alternatives with much more efficient inference.

show abstract

Section: Related Workmentioning

confidence: 99%

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

Zhou

Yang

Liu

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Inference: In the inference time, the pixel accuracy (pixAcc), mean accuracy, and mean pixel Intersection-overunion (mIoU) are leveraged as the evaluation metrics for the LIP dataset, pixel accuracy, precision, recall, and F-1 scores for the ATR dataset, and mIoU for the CIHP dataset. Similar to [10,23,32,41], we averaged the results using 3-scale image pyramids of different scales [0.75, 1.0, 1.25] and flipping for further performance improvement.…”

Section: Experimental Settingsmentioning

confidence: 99%

CDGNet: Class Distribution Guided Network for Human Parsing

Liu¹,

Choi²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

The objective of human parsing is to partition a human in an image into constituent parts. This task involves labeling each pixel of the human image according to the classes. Since the human body comprises hierarchically structured parts, each body part of an image can have its sole position distribution characteristics. Probably, a human head is less likely to be under the feet, and arms are more likely to be near the torso. Inspired by this observation, we make instance class distributions by accumulating the original human parsing label in the horizontal and vertical directions, which can be utilized as supervision signals. Using these horizontal and vertical class distribution labels, the network is guided to exploit the intrinsic position distribution of each class. We combine two guided features to form a spatial guidance map, which is then superimposed onto the baseline network by multiplication and concatenation to distinguish the human parts precisely. We conducted extensive experiments to demonstrate the effectiveness and superiority of our method on three well-known benchmarks: LIP, ATR, and CIHP databases.

show abstract

“…Since the human body contains highly structural information, many previous methods enhance the pixel-level representations with well-designed architectures that can capture the global context cues, such as global context embeddings [4], [26], generative adversarial networks [25], [27], and recurrent models [28], [29]. Apart from pixel-level semantics, human part classes naturally have rich structural semantics, hence, many works model the body part correlations explicitly by building, e.g., graph neural networks [30], [31], tree-like topology information passing architectures [32], [33], and hierarchical human structures [34], [35], [36]. Another direction is exploiting common semantics among different human-centric tasks, e.g., pose estimation and keypoint detection [6], [37], [37], [38], [39], [40], [41] or other prior human semantics, e.g., edge information or human contour [30], [42].…”

Section: Human Parsingmentioning

confidence: 99%

“…PPNet. We also compare our method with PPNet [35], which employs part-aware prototypes. Since one human scene contains multiple human classes and we lack the global semantic class information described in PPNet, we have to remove the semantic branch in [35].…”

Section: Contendersmentioning

confidence: 99%

End-to-end One-shot Human Parsing

He¹,

Zhang²,

Zhuang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Previous human parsing models are limited to parsing humans into pre-defined classes, which is inflexible for applications that need to handle new classes. In this paper, we define a new one-shot human parsing (OSHP) task that requires parsing humans into an open set of classes defined by any test example. During training, only base classes are exposed, which only overlap with part of test-time classes. To address three main challenges in OSHP, i.e., small sizes, testing bias, and similar parts, we devise a novel Endto-end One-shot human Parsing Network (EOP-Net). Firstly, an end-to-end human parsing framework is proposed to mutually share semantic information with different granularities and help recognize the small-size human classes. Then, we devise two collaborative metric learning modules to learn representative prototypes for base classes, which can quickly adapt to unseen classes and mitigate the testing bias. Moreover, we empirically find that robust prototypes empower feature representations with higher transferability to the novel concepts, hence, we propose to adopt momentum-updated dynamic prototypes generated by gradually smoothing the training time prototypes and employ contrastive loss at the prototype level. Experiments on three popular benchmarks tailored for OSHP demonstrate that EOP-Net outperforms representative one-shot segmentation models by large margins, which serves as a strong benchmark for further research on this new task. The source code will be made publicly available.

show abstract

Part-Aware Context Network for Human Parsing

Cited by 56 publications

References 57 publications

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

CDGNet: Class Distribution Guided Network for Human Parsing

End-to-end One-shot Human Parsing

Contact Info

Product

Resources

About