Recognizing Rotated Faces From Frontal and Side Views: An Approach Toward Effective Use of Mugshot Databases

Abstract

Abstract-Mug shot photography has been used to identify criminals by the police for more than a century. However, the common scenario of face recognition using frontal and side-view mug shots as gallery remains largely uninvestigated in computerized face recognition across pose. This paper presents a novel appearance-based approach using frontal and sideface images to handle pose variations in face recognition, which has great potential in forensic and security applications involving police mugshot databases. Virtual views in different poses are generated in two steps: 1) shape modelling and 2) texture synthesis. In the shape modelling step, a multilevel variation minimization approach is applied to generate personalized 3-D face shapes. In the texture synthesis step, face surface properties are analyzed and virtual views in arbitrary viewing conditions are rendered, taking diffuse and specular reflections into account. Appearance-based face recognition is performed with the augmentation of synthesized virtual views covering possible viewing angles to recognize probe views in arbitrary conditions. The encouraging experimental results demonstrated that the proposed approach by using frontal and side-view images is a feasible and effective solution to recognizing rotated faces, which can lead to a better and practical use of existing forensic databases in computerized human face-recognition applications.

A Novel Approach to Mugshot Based Arbitrary View Face Recognition

“…4, which again proves the effectiveness of the proposed method in recognizing face images of large pose angles. The rank-1 accuracy of the proposed method is better than the one in [11] especially for large poses (i.e., c31, c14, c25, c02). The recognition rate of the method in [11] for pose c31 is 80%, whereas our recognition rate reaches 93%.…”

“…The rank-1 accuracy of the proposed method is better than the one in [11] especially for large poses (i.e., c31, c14, c25, c02). The recognition rate of the method in [11] for pose c31 is 80%, whereas our recognition rate reaches 93%. The rank-1 accuracy of the proposed method is firmly higher than the other one across poses.…”

“…We also compare the proposed method with the one in [11]. It is the only published research work we are aware of that reported mugshot-based face recognition accuracy on the CMU PIE database.…”

“…It is the only published research work we are aware of that reported mugshot-based face recognition accuracy on the CMU PIE database. For a fair comparison, following the protocol in [11], we choose pose c27 as the frontal face, and c22 and c34 as the left profile face and right profile face, respectively. The remaining 10 poses of 68 persons are considered as probe images.…”

“…When both frontal and profile face images are available in the gallery, in order to utilize the profile face images to better recognize arbitrary view faces, Zhang et al [11] first reconstructed 3D face shapes from the frontal and profile face images, and then recovered facial textures by fitting the Phong reflection model based on the obtained 3D face shapes and the assumed lighting direction prior. They generated synthetic face images of various poses for each enrolled subject to enlarge the gallery.…”