Hierarchical Multi-Task Network For Race, Gender and Facial Attractiveness Recognition

Xu, Lu; Fan, Heng; Xiang, Jinhai

doi:10.1109/icip.2019.8803614

Cited by 27 publications

(25 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…contributing to attractiveness, we tested it on the SCUT-FBP5500 dataset (Liang et al, 2018) with attractiveness scores, which has been used for emerging studies of facial attractiveness analysis in both computer vision (Xu et al, 2019;Shi et al, 2019) and the psychological community (Zhao et al, 2019). The dataset contains 5,500 face images.…”

Section: Resultsmentioning

confidence: 99%

“…For the female faces, the average correlation coefficient between the scores of Neuron-HAF and the attractiveness scores was r = 0.7583±0.0198, all p <0.00001 (seeFigure 3a), whereas the average correlation coefficient between the scores of Neuron-UAF and the attractiveness scores was r = −0.5655 ± 0.0217, all p < 0.00001. For the male faces, the average correlation coefficient between the scores of Neuron-HAM and the attractiveness scores was r = 0.6653 ± 0.0161, all p < 0.00001 (seeFigure 3b), whereas the average correlation coefficient between the scores of Neuron-UAM and the attractiveness scores was r = −0.6221 ± 0.0202, all p < 0.00001 Xu et al (2019). found that the handcrafted feature-based models can predict attractiveness scores with Pearson correlations of [0.6, 0.7] whereas the DNN-based models can predict attractiveness scores with Pearson correlations of [0.8,0.9].…”

mentioning

confidence: 96%

See 1 more Smart Citation

Putative Ratios of Facial Attractiveness in a Deep Neural Network

Song¹,

Liang²,

Kumada³

et al. 2020

Preprint

View full text Add to dashboard Cite

Empirical evidence has shown that there is an ideal arrangement of facial features (ideal ratios) that can optimize the attractiveness of a person's face. These putative ratios define facial attractiveness in terms of spatial relations and provide important rules for measuring the attractiveness of a face. In this paper, we show that a deep neural network (DNN) model can learn putative ratios based only on categorical annotation when no annotated facial features for attractiveness are explicitly given. To this end, we conducted three experiments. In Experiment 1, we trained a DNN model to recognize facial attractiveness using four category-specific neurons (female/male $\times$ high/low attractiveness). In Experiment 2, face-like images were generated by reversing the DNN model (e.g., deconvolution). These images depict the intuitive attributes of the four categories of facial attractiveness and reveal certain consistencies with reported evidence on the putative ratios of facial attractiveness. In Experiment 3, simulated psychophysical experiments on facial images with varying ratios of features reveal changes in the activity of the category-specific neurons that are remarkably similar to those of human judgements reported in a previous study. These results show that the trained DNN model can learn putative ratios as key features for the representation of facial attractiveness. These findings advance our understanding of facial attractiveness and high-level human perception.

show abstract

Section: Resultsmentioning

confidence: 99%

mentioning

confidence: 96%

Putative Ratios of Facial Attractiveness in a Deep Neural Network

Song¹,

Liang²,

Kumada³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…This multi-task framework improves the performance of both tasks and alleviates the over-fitting problem in training the network. Xu et al [31] also proposed a multi-task deep learning framework for facial attractiveness, gender, and race prediction, achieving state-of-the-art performance. Zhou et al [32] presented a system for analyzing trends in perceived attractiveness of Chinese males at different times.…”

Section: Related Workmentioning

confidence: 99%

Assessing Facial Symmetry and Attractiveness using Augmented Reality

Wei

McCay

et al. 2021

Pattern Anal Applic

View full text Add to dashboard Cite

Facial symmetry is a key component in quantifying the perception of beauty. In this paper, we propose a set of facial features computed from facial landmarks which can be extracted at a low computational cost. We quantitatively evaluated the proposed features for predicting perceived attractiveness from human portraits on four benchmark datasets (SCUT-FBP, SCUT-FBP5500, FACES and Chicago Face Database). Experimental results showed that the performance of the proposed features is comparable to those extracted from a set with much denser facial landmarks. The computation of facial features was also implemented as an augmented reality (AR) app developed on Android OS. The app overlays four types of measurements and guidelines over a live video stream, while the facial measurements are computed from the tracked facial landmarks at run time. The developed app can be used to assist plastic surgeons in assessing facial symmetry when planning reconstructive facial surgeries.

show abstract

“…Recently, there is a growing wave of research in multi-task learning, by which diverse computer vision tasks are solved. Lu et al [35] proposed a hierarchical multi-task network (HMTNet), which can achieve the effect of simultaneously identifying a person's gender, race, and facial attractiveness from a given portrait image. A DMTL model for keypoint detection, face detection, posture estimation was proposed by [36].…”

Section: B Multi-task Transfer Learningmentioning

confidence: 99%

2M BeautyNet: Facial Beauty Prediction Based on Multi-Task Transfer Learning

Gan

Scotti

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Facial beauty prediction (FBP) has become an emerging area in the field of artificial intelligence. However, the lacks of data and accurate face representation hinder the development of FBP. Multi-task transfer learning can effectively avoid over-fitting, and utilize auxiliary information of related tasks to optimize the main task. In this paper, we present a network named Multi-input Multi-task Beauty Network (2M BeautyNet) and use transfer learning to predict facial beauty. In the experiment, beauty prediction is the main task, and gender recognition is the auxiliary. For multi-task training, we employ multitask loss weights automatic learning strategy to improve the performance of FBP. Finally, we replace the softmax classifier with a random forest. We conduct experiments on the Large Scale Facial Beauty Database (LSFBD) and SCUT-FBP5500 database. Results show that our method has achieved good results on LSFBD, the accuracy of FBP is up to 68.23%. Our 2M BeautyNet structure is suitable for multiple inputs of different databases. INDEX TERMS Facial beauty prediction, multi-task transfer learning, multi-input multi-output network, multi-task loss weight automatic learning strategy.

show abstract

Hierarchical Multi-Task Network For Race, Gender and Facial Attractiveness Recognition

Cited by 27 publications

References 22 publications

Putative Ratios of Facial Attractiveness in a Deep Neural Network

Putative Ratios of Facial Attractiveness in a Deep Neural Network

Assessing Facial Symmetry and Attractiveness using Augmented Reality

2M BeautyNet: Facial Beauty Prediction Based on Multi-Task Transfer Learning

Contact Info

Product

Resources

About