The FaceSketchID System: Matching Facial Composites to Mugshots

Abstract

Abstract-Facial composites are widely used by law enforcement agencies to assist in the identification and apprehension of suspects involved in criminal activities. These composites, generated from witness descriptions, are posted in public places and in the media with the hope that some viewers will provide tips about the identity of the suspect. This method of identifying suspects is slow, tedious, and may not lead to the timely apprehension of a suspect. Hence, there is a need for a method that can automatically and efficiently match facial composites to large police mugshot databases. Because of this requirement, facial composite recognition is an important topic for biometrics researchers. While substantial progress has been made in nonforensic facial composite (or viewed composite) recognition over the past decade, very little work has been done using operational composites relevant to law enforcement agencies. Furthermore, no facial composite to mugshot matching systems have been documented that are readily deployable as standalone software. Thus, the contributions of this paper include: (i) an exploration of composite recognition use cases involving multiple forms of facial composites, (ii) the FaceSketchID System, a scalable and operationally deployable software system that achieves stateof-the-art matching accuracy on facial composites using two algorithms (holistic and component-based), and (iii) a study of the effects of training data on algorithm performance. We present experimental results using a large mugshot gallery that is representative of a law enforcement agency's mugshot database. All results are compared against three state-of-the-art commercial-off-the-shelf (COTS) face recognition systems.

Fusion of Intra- and Inter-modality Algorithms for Face-Sketch Recognition

“…Although most approaches have focused on reducing the modality gap, intermodality methods are being given more attention lately especially by the Biometrics Research Group at Michigan State University [4,7,9,10]. These approaches are based on the extraction of Multi-scale Local Binary Pattern (MLBP) and Scale-Invariant Feature Transform (SIFT) descriptors from patches to directly match sketches and photos in terms of their feature representations.…”

“…These approaches are based on the extraction of Multi-scale Local Binary Pattern (MLBP) and Scale-Invariant Feature Transform (SIFT) descriptors from patches to directly match sketches and photos in terms of their feature representations. The recently proposed FaceSketchID system [10] combines the component-based approach proposed in [4], in which MLBP codes were extracted for each facial component, with the holistic-based heterogeneous face recognition algorithm in [9] to obtain a system with performance that was shown to exceed that of either of the two techniques individually. In other words, the fusion of local and global inter-modality algorithms yielded improvements in performance.…”

“…Recollections of eye-witnesses and victims are important to assist law enforcement agencies in identifying and apprehending suspects, especially when there is lack of evidence at the crime scene [5,10]. Facial composite sketches created using witness descriptions are then disseminated to the public so that any persons recognising the suspect in the sketch may come forward with information that will hopefully lead to an arrest.…”

“…Facial composite sketches created using witness descriptions are then disseminated to the public so that any persons recognising the suspect in the sketch may come forward with information that will hopefully lead to an arrest. However, this process is time-consuming, not guaranteed to be successful and inefficient since resources such as mugshot photo galleries available at law enforcement agencies cannot be utilised [10]. Hence, automated methods are required to match sketches with photos.…”

Suspect identification based on descriptive facial attributes

“…Recognition results were generated on the test data by using the human provided attributes as query/probe, and the features automatically extracted from the photographs for the target/gallery. The baseline used was the MSU FaceSketchID sketch recognition algorithm [12], which achieves state of the art accuracy in sketch recognition. The training data was used to train the discriminative subspaces in the baseline sketch recognition algorithm.…”