A computational approach to investigating facial attractiveness factors using geometric morphometric analysis and deep learning

Abstract

Numerous studies discuss the features that constitute facial attractiveness. In recent years, computational research has received attention because it can examine facial features without relying on prior research hypotheses. This approach uses many face stimuli and models the relationship between physical facial features and attractiveness using methods such as geometric morphometrics and deep learning. However, studies using each method have been conducted independently and have technical and data-related limitations. It is also difficult to identify the factors of actual attractiveness perception using only computational methods. In this study, we examined morphometric features important for attractiveness perception through geometric morphometrics and impression evaluation. Furthermore, we used deep learning to analyze important facial features comprehensively. The results showed that eye-related areas are essential in determining attractiveness and that different racial groups contribute differently to the impact of shape and skin information on attractiveness. The approach used in this study will contribute toward understanding facial attractiveness features that are universal and diverse, extending psychological findings and engineering applications.

Exploring Predictors of Facial Impressions: An Integrated Approach Utilizing VAE, Grad-CAM, and GPT-4V

“…We constructed a model that generates facial images using only a small amount of data. During the generation process, as the attractiveness score increased, morphological features such as suspended eyes and raised eyebrows changed, and the trends of these features were consistent with those reported in a previous study (Sano & Kawabata, 2023). The larger the input for the dominance score, the larger the face, which corresponds to the relationship between the face width and dominance (Carré & McCormick, 2008).…”

“…Visual inspections of the results showed that morphological features such as contour sharpness and eyebrow angle changed when the impression of attractiveness was manipulated as the input value. Features such as sharply angled contours and elevated eyebrows were confirmed as attractive, which was consistent with the previous study (Sano & Kawabata, 2023). Additionally, manipulating the dominance impression as the input value resulted in changes in the morphological feature of face width; a previous report has also noted a relationship between facial width and aggression (Carré & McCormick, 2008).…”

“…Additionally, the ReLU and linear output activation functions were employed. The model structure was the same as that used in a previous study (Sano & Kawabata, 2023), including the Adam optimizer, a learning rate of 0.001, a batch size of 16, and 300 training epochs. The original images were compared with the augmented data through three-point cross-validation.…”

“…The original images were compared with the augmented data through three-point cross-validation. Based on previous studies (Xu et al, 2017;Sano 2022;Sano 2023;Sano & Kawabata, 2023), we used the Pearson correlation between the impression labels of the images and predicted results as a metric. In addition, we built a model trained on all original and augmented data and visualized the facial areas important for predictions using Grad-CAM, which is an excellent model for confirming CNN model predictors.…”

“…For example, Sano (2022;2023) revealed that sexual dimorphism is an important factor for facial attractiveness using CNN models and gradient-weighted class activation mapping (Grad-CAM; Selvaraju et al, 2017), which is used for visualizing the basis of judgments. A subsequent study showed that the eye region is critical for attractiveness (Sano & Kawabata, 2023). These DL methods, as well as computational methods such as morphometrics (Windhager et al, 2018;; image statistical analysis (Arce-Lopera et al, 2013;Fink et al, 2001); and 3D modeling (Nakamura & Watanabe, 2019;Oh et al, 2023;Said & Todorov, 2011) can comprehensively explore critical facial features.…”