Describe Me If You Can! Characterized Instance-Level Human Parsing

Abstract

Several computer vision applications such as person search or online fashion rely on human description. The use of instance-level human parsing (HP) is therefore relevant since it localizes semantic attributes and body parts within a person. But how to characterize these attributes? To our knowledge, only some single-HP datasets describe attributes with some color, size and/or pattern characteristics. There is a lack of dataset for multi-HP in the wild with such characteristics. In this article, we propose the dataset CCIHP based on the multi-HP dataset CIHP, with 20 new labels covering these 3 kinds of characteristics. 1 In addition, we propose HPTR, a new bottom-up multi-task method based on transformers as a fast and scalable baseline. It is the fastest method of multi-HP state of the art while having precision comparable to the most precise bottom-up method. We hope this will encourage research for fast and accurate methods of precise human descriptions.

MoRF: Mobile Realistic Fullbody Avatars from a Monocular Video

“…We retrain Graphonomy [13] segmentation network on CCIHP dataset [31], which includes accessories like watches, belts, and glasses, etc. Meanwhile the publicly available Graphonomy trained on CIHP dataset predicts those as background.…”

Can Human Attribute Segmentation be More Robust to Operational Contexts Without New Labels?

“…The goal is to improve the model robustness by adapting it to the target domain without any annotation, despite its discrepancy from the source dataset. Therefore, we use HPTR [1] as our base architecture. The original method performs human detection and instance segmentation, as well as attribute and characteristic segmentation.…”

“…Yet, some bottom-up methods are not fast enough due to their expensive post-processing [10] or heavy GAN architectures [2]. Currently, HPTR [1] (Human Parsing with TRansformers) is the fastest bottom-up approach while having comparable performance with other state-of-the-art (SOTA) methods. It is an end-to-end multi-task approach based on the detector DETR [11], jointly providing human detection and instance segmentation, in addition to predicting human attributes and their characteristics (size, pattern, color).…”

“…2), a first solution studying UDA for HAS. It is based on HPTR [1], the fastest and scalable HP method with competitive performance. We study how to improve its robustness to contexts by combining two label-free paradigms.…”

“…The effectiveness of the proposed approach is demonstrated through extensive ablation experiments on a new proposed benchmark for UDA-HAS. Our contributions can be formulated as follows: (1) We propose UDA-HPTR, an UDA method for HAS by jointly adopting self-supervised and semi-supervised approaches to the specific task of HAS. To the best of our knowledge, this is the first attempt that combines these two paradigms for a better UDA, and that studies UDA for HAS.…”