Self-Attention-Masking Semantic Decomposition and Segmentation for Facial Attribute Manipulation

Xia, Xuan; Yu, Fengqi; Li, Nan; Qu, Yansong; Zhang, Jia-Jia; Zhu, Chengguang

doi:10.1109/access.2020.2974239

Cited by 6 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many studies for human hair segmentation have impacted various face-related tasks, such as facial recognition and style translation. Particularly, hair segmentation is required for successful style transformation in the field of face-centered style transfer, such as face translation [42] and face editing [43], [44]. Early research focused primarily on the prediction of hair masks by only using color, spatial, and frequency information [45], [46], [47], [48].…”

Section: Human Hair Segmentationmentioning

confidence: 99%

Fine-Grained Human Hair Segmentation Using a Text-to-Image Diffusion Model

Kim,

Lee,

Yoo

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Human hair segmentation is essential for face recognition and for achieving natural transformation of style transfer. However, it remains a challenging task due to the diverse appearances and complex patterns of hair in image. In this study, we propose a novel method utilizing diffusion-based generative models, which have been extensively researched in recent times, to effectively capture and to finely segment human hair. In diffusion-based models, an internal visual representation during the denoising process contains pixel-level rich information. Inspired by this aspect, we introduce diffusion-based models for segmenting fine-grained human hair. Specifically, we extract the representation from the diffusion-based models, which contains pixel-level semantic information, and then train a segmentation network using it. Particularly, to more finely segment human hair, our approach employs the representation from a text-toimage diffusion model, conditioned on text information, to extract more relevant information for human hair, thereby predicting detailed hair masks. To validate our method, we conducted experiments on three distinct hair-related datasets with unique characteristics: Figaro-1k, CelebAMask-HQ, and Face Synthetics. The experimental results show the improved performance of our proposed method across all three datasets, outperforming existing methods in terms of mIoU (mean intersection over union), accuracy, precision, and F1-score. This is particularly evident in its ability to accurately capture and finely segment human hair from background and non-hair elements. This demonstrates the effectiveness of our method in accurately and finely segmenting human hair with complex characteristics. Our research contributes not only to the fine-grained segmentation of human hair but also to the application of generative models in semantic segmentation tasks. We hope that the proposed method will be applied for detailed semantic segmentation in various fields in the future.

show abstract

Section: Human Hair Segmentationmentioning

confidence: 99%

Fine-Grained Human Hair Segmentation Using a Text-to-Image Diffusion Model

Kim,

Lee,

Yoo

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Self-attention-masking model [182] is also based on the idea of semantic segmentation and decomposition of the face region. As shown in Figure 46, the generator of this model consists of two parts: G c which generates a modified image as color mask and G a which is responsible to produce an attention mask.…”

Section: Sun Et Al Proposed Mask-adversarial Autoencoder (M-aae)mentioning

confidence: 99%