Asian Female Facial Beauty Prediction Using Deep Neural Networks via Transfer Learning and Multi-Channel Feature Fusion

Abstract

Facial beauty plays an important role in many fields today, such as digital entertainment, facial beautification surgery and etc. However, the facial beauty prediction task has the challenges of insufficient training datasets, low performance of traditional methods, and rarely takes advantage of the feature learning of Convolutional Neural Networks. In this paper, a transfer learning based CNN method that integrates multiple channel features is utilized for Asian female facial beauty prediction tasks. Firstly, a Large-Scale Asian Female Beauty Dataset (LSAFBD) with a more reasonable distribution has been established. Secondly, in order to improve CNN's self-learning ability of facial beauty prediction task, an effective CNN using a novel Softmax-MSE loss function and a double activation layer has been proposed. Then, a data augmentation method and transfer learning strategy were also utilized to mitigate the impact of insufficient data on proposed CNN performance. Finally, a multi-channel feature fusion method was explored to further optimize the proposed CNN model. Experimental results show that the proposed method is superior to traditional learning method combating the Asian female FBP task. Compared with other state-of-the-art CNN models, the proposed CNN model can improve the rank-1 recognition rate from 60.40% to 64.85%, and the pearson correlation coefficient from 0.8594 to 0.8829 on the LSAFBD and obtained 0.9200 regression prediction results on the SCUT dataset. INDEX TERMS Convolutional neural network (CNN), double activation layer, facial beauty prediction (FBP), feature fusion, softmax-MSE loss, transfer learning.

Assessing Facial Symmetry and Attractiveness using Augmented Reality

“…Our method outperforms the comparable methods on beauty score prediction and beauty score classification, including the method of Xu et al [31], which achieved 0.2501 MAE and 0.3263 RSME on SCUT-FBP5500 using Hierarchical Multi-task Network (HMT-Net). Other deep learning-based methods (such as presented in [50]) additionally use color and texture-based features, i.e., they cannot be directly compared to our method, which is based on geometric features of a face.…”

Facial Beauty Prediction and Analysis based on Deep Convolutional Neural Network: A Review

“…Zhai et al [48] proposed using a transfer learning-based CNN technique that allows various channel features to predict Asian female facial beauty. In the first instance, LSAFBD was created with a more reasonable distribution.…”

Facial Beauty Prediction via Local Feature Fusion and Broad Learning System

“…In recent years, some new methods for studying facial beauty [20] have sprung up in neural networks and deep learning. Gan et al [21] employed the deep convolutional network model for feature learning with a correlation of 0.739 while Jie et al [22] established a psychologically stimulated convolutional neural network model to predict facial beauty, achieving a correlation of 0.87.…”

“…Deep learning has fully displayed its strengths in the field of FBP in the past decade [5][6][7]. Compared with conventional machine learning approaches, deep learning networks can automatically extract higher-level features from facial data [8]. Although deep learning has realized excellent prediction results in FBP, its imperfections [9] can never be underestimated: (1) time-consuming and costly training process due to a great deal of parameters; (2) poor model generalization [10] arisen from deep learning's greater sensitivity to image texture features than to image shape features; (3) huge calculation needed resulting from the complex structure of deep learning network.…”