Unconstrained Face Recognition: Identifying a Person of Interest From a Media Collection

Abstract

Abstract-As face recognition applications progress from constrained sensing and cooperative subjects scenarios (e.g., driver's license and passport photos) to unconstrained scenarios with uncooperative subjects (e.g., video surveillance), new challenges are encountered. These challenges are due to variations in ambient illumination, image resolution, background clutter, facial pose, expression, and occlusion. In forensic investigations where the goal is to identify a "person of interest," often based on low quality face images and videos, we need to utilize whatever source of information is available about the person. This could include one or more video tracks, multiple still images captured by bystanders (using, for example, their mobile phones), 3D face models constructed from image(s) and video(s), and verbal descriptions of the subject provided by witnesses. These verbal descriptions can be used to generate a face sketch and provide ancillary information about the person of interest (e.g., gender, race, and age). While traditional face matching methods generally take a single media (i.e., a still face image, video track, or face sketch) as input, our work considers using the entire gamut of media as a probe to generate a single candidate list for the person of interest. We show that the proposed approach boosts the likelihood of correctly identifying the person of interest through the use of different fusion schemes, 3D face models, and incorporation of quality measures for fusion and video frame selection.

The FaceSketchID System: Matching Facial Composites to Mugshots

“…The options for localizing a component on the face are limited, and therefore achieving the desired alignment of components is also challenging. In total, 75 software-generated [28]. The subject shown is MarieJosee Croze.…”

“…We investigate the possibility of using this form of facial composite to improve upon the matching performance of COTS matchers using a set of (surveillance composite, mugshot) pairs provided to us by the PCSO (Fig. 4) and a set of pairs from [28] (Fig. 5).…”

“…We report the retrieval rank achieved using the most accurate composite and the most accurate frame from the surveillance video for each subject. We include a surveillance composite depicting Tamerlan Tsarnaev (the older brother alleged to be involved in the Boston Marathon bombing) [28] and report the FaceSketchID System's performance when matching the composite to photographs from [42]. We also evaluate the performance of the FaceSketchID System on a set of 21 surveillance composites created while viewing videos from the YouTube Faces (YTF) [43] database for [28].…”

“…We include a surveillance composite depicting Tamerlan Tsarnaev (the older brother alleged to be involved in the Boston Marathon bombing) [28] and report the FaceSketchID System's performance when matching the composite to photographs from [42]. We also evaluate the performance of the FaceSketchID System on a set of 21 surveillance composites created while viewing videos from the YouTube Faces (YTF) [43] database for [28]. In [28], the mated photographs used for these composites are from the Labeled Faces in the Wild (LFW) [44] database.…”

Improving Cloud Security by the Application of De-Duplication

“…The protected preparing of individual information in the cloud speaks to an enormous test. Appropriation of security upgrading advances to help such exercises in the cloud will rely on the presence of uniform methods for taking care of individual information at the worldwide level and on specialized gauges which can show consistence with lawful and administrative frameworks [11]. The computational asset force of the progression can be constraints of information de-duplication.…”