Deep Learning Face Attributes in the Wild

Abstract

Predicting face attributes in the wild is challenging due to complex face variations. We propose a novel deep learning framework for attribute prediction in the wild. It cascades two CNNs, LNet and ANet, which are finetuned jointly with attribute tags, but pre-trained differently. LNet is pre-trained by massive general object categories for face localization, while ANet is pre-trained by massive face identities for attribute prediction. This framework not only outperforms the state-of-the-art with a large margin, but also reveals valuable facts on learning face representation.(1) It shows how the performances of face localization (LNet) and attribute prediction (ANet) can be improved by different pre-training strategies. (2) It reveals that although the filters of LNet are fine-tuned only with imagelevel attribute tags, their response maps over entire images have strong indication of face locations. This fact enables training LNet for face localization with only image-level annotations, but without face bounding boxes or landmarks, which are required by all attribute recognition works. (3) It also demonstrates that the high-level hidden neurons of ANet automatically discover semantic concepts after pretraining with massive face identities, and such concepts are significantly enriched after fine-tuning with attribute tags. Each attribute can be well explained with a sparse linear combination of these concepts.

A Correlational Neural Network for Gender Classification

“…In order to demonstrate the validity of the proposed CorNN for face gender classification, we compare it with a CNN with the same structure on nine human face databases: ORL, Georgia Tech, FERET [15], Extended Yale B (EYB) [16], AR [17], Faces94, LFW [18], MORPH and CelebFaces+ [19]. The examples of face images from these databases are shown in Fig.…”

From Face Images and Attributes to Attributes

“…Convolutional Neural Networks (CNNs) proved their effectiveness on many computer vision tasks such as classification and end-to-end mappings [8][9][10]. The same is valid for attribute prediction where the recent top solutions rely on deep learning for representations and/or for direct attribute prediction [1,6,[11][12][13][14]. We observe that generally the methods treat the prediction of each attribute individually starting from the face image even if there are clear dependencies between some attributes ('males' are unlikely to wear 'earrings' or 'lipsticks', 'females' to be 'bold' and have a 'goatee').…”

“…Attribute prediction is a challenging problem and while for biometrics the research spans over decades, the other attributes have been investigated less and, moreover, most of the recent research treats each attribute individually as a prediction from face image domain to the attribute label. The recent release of large-scale attribute datasets such as CelebA [6] (with face attributes) and Facebook BIG5 (with personality traits and 'likes') [7] fueled the recent advances and the interest resurgence for attributes as research topic. Convolutional Neural Networks (CNNs) proved their effectiveness on many computer vision tasks such as classification and end-to-end mappings [8][9][10].…”

Real‐time AdaBoost cascade face tracker based on likelihood map and optical flow

“…The complexity of the CNN increases from stage to stage, in a way that quick rejection of non-face windows is done at the beginning, then a more complex CNN is used in the last stage to refine the results. The training databases employed include WIDER FACE [71] and CelebA [19] databases, while evaluation results reported in [7] were obtained using Face Detection Data Set and Benchmark (FDDB) [22], Annotated Facial Landmarks in the Wild (AFLW) [23] and WIDER FACE databases. We use the code released by the authors 1 for our experiments, which is implemented using Caffe.…”

“…quick rejection of background regions) has been applied to features learned by Convolutional Neural Networks (CNN) [18], and the amount of research works on face detection making use of CNNs is exploding, e.g. [7,19,20,21], inspired by the remarkable recent success of CNNs in many computer vision tasks. A drawback of these approaches, and of many approaches for unconstrained face detection, is that they usually need a considerable amount of annotated training data, apart from being computationally expensive [7].…”