Face recognition using deep multi-pose representations

AbdAlmageed, Wael; Wu, Yue; Rawls, Stephen; Har-El, Shai; Hassner, Tal; Masi, Iacopo; Choi, Jongmoo; Lekust, Jatuporn; Kim, Jungyeon; Natarajan, Prem; Nevatia, Ram; Medioni, Gérard

doi:10.1109/wacv.2016.7477555

Cited by 136 publications

(73 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For face identification, a major accomplishment of DCNNs is their visual robustness to changes in viewpoint, illumination, expression, and appearance (AbdAlmageed et al., ; Chen, Patel, & Chellappa, ; Ranjan, Sankaranarayanan, Castillo, & Chellappa, ; N. Zhang et al., ). This is a consequence of the fact that DCNN architectures for face identification are trained on large numbers of identities and use multiple, variable images of each identity.…”

Section: Introductionmentioning

confidence: 99%

Social Trait Information in Deep Convolutional Neural Networks Trained for Face Identification

Parde

Castillo

et al. 2019

Cognitive Science

View full text Add to dashboard Cite

Faces provide information about a person's identity, as well as their sex, age, and ethnicity. People also infer social and personality traits from the face — judgments that can have important societal and personal consequences. In recent years, deep convolutional neural networks (DCNNs) have proven adept at representing the identity of a face from images that vary widely in viewpoint, illumination, expression, and appearance. These algorithms are modeled on the primate visual cortex and consist of multiple processing layers of simulated neurons. Here, we examined whether a DCNN trained for face identification also retains a representation of the information in faces that supports social‐trait inferences. Participants rated male and female faces on a diverse set of 18 personality traits. Linear classifiers were trained with cross validation to predict human‐assigned trait ratings from the 512 dimensional representations of faces that emerged at the top‐layer of a DCNN trained for face identification. The network was trained with 494,414 images of 10,575 identities and consisted of seven layers and 19.8 million parameters. The top‐level DCNN features produced by the network predicted the human‐assigned social trait profiles with good accuracy. Human‐assigned ratings for the individual traits were also predicted accurately. We conclude that the face representations that emerge from DCNNs retain facial information that goes beyond the strict limits of their training.

show abstract

Section: Introductionmentioning

confidence: 99%

Social Trait Information in Deep Convolutional Neural Networks Trained for Face Identification

Parde

Castillo

et al. 2019

Cognitive Science

View full text Add to dashboard Cite

show abstract

“…The higher the values are, the better the performance is. We compare four pooling methods including scoremax [25], score-average [26], [28], feature-average [11], [12], [13] and kNN-average pooling based on three different metrics. ROC curves are drawn for each setting in Fig.…”

Section: B Ijb-a Evaluationmentioning

confidence: 99%

“…In this case, besides the old problem of how to extract invariable and discriminative features, finding a solution to match two sets of media is also challenging. Most of the existing methods until now mainly include feature pooling [11], [12], [13] and score pooling [25], [26], [28], where the former suggests aggregating features over all images in a set while the latter aggregates the pair-wise similarity scores of two compared sets. However, neither of the two measurements performs well when the faces of certain subjects are with many extreme poses or other variations, which occur frequently in reality.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, the authors tried two kinds of max pooling methods within a set: score-max pooling and feature-max pooling. Alternatively, [26] exploited 3D rendering to generate multiple face poses from the input image and then produced multiple pose-specific features by several pose-specific DCNN models. They simply adopted a direct feature-to-feature comparison in a pair-wise fashion and used the softmax weights to fuse all the scores for a set.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unconstrained Face Recognition Using a Set-to-Set Distance Measure on Deep Learned Features

Zhao

Han

Shao

2018

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Abstract-Recently considerable efforts have been dedicated to unconstrained face recognition, which requires to identify faces "in the wild" for a set of images and/or video frames captured without human intervention. Unlike traditional face recognition that compares one-to-one media (either a single image or a video frame) only, we encounter a problem of matching sets with heterogeneous contents containing both images and videos. In this paper, we propose a novel Set-to-Set (S2S) distance measure to calculate the similarity between two sets with the aim to improve the recognition accuracy for faces with real-world challenges such as extreme poses or severe illumination conditions. Our S2S distance adopts the kNN-average pooling for the similarity scores computed on all the media in two sets, making the identification far less susceptible to the poor representations (outliers) than traditional feature-average pooling and score-average pooling. Furthermore, we show that various metrics can be embedded into our S2S distance framework, including both predefined and learned ones. This allows to choose the appropriate metric depending on the recognition task in order to achieve the best results. To evaluate the proposed S2S distance, we conduct extensive experiments on the challenging set-based IJB-A face dataset, which demonstrate that our algorithm achieves the stateof-the-art results and is clearly superior to the baselines including several deep learning based face recognition algorithms.

show abstract

“…However, the experiments in [10] have shown that it has got a bad performance in AR sunglass image set, only achieving less than 50% recognition rate. Wael AbdAlmageed et al [13] use several pose specific deep convolution neural network (CNN) models to generate multiple pose-specific features. Jun-Cheng Chen et al [14] present an algorithm for unconstrained face verification based on deep convolution features, and obtain a good recognition result on the occasion with pose variations.…”

Section: Related Workmentioning

confidence: 99%