Sketch based face recognition: Forensic vs. composite sketches

Abstract

Facial sketches are widely used by law enforcement agencies to assist in the identification and apprehension of suspects involved in criminal activities. Sketches used in forensic investigations are either drawn by forensic artists (forensic sketches) or created with computer software (composite sketches) following the verbal description provided by an eyewitness or the victim. These sketches are posted in public places and in media in hopes that some viewers will provide tips about the identity of the suspect. This method of identifying suspects is slow and tedious and may not lead to apprehension of the suspect. Hence, there is a need for a method that can automatically and quickly match facial sketches to large police mugshot databases. We address the problem of automatic facial sketch to mugshot matching and, for the first time, compare the effectiveness of forensic sketches and composite sketches. The contributions of this paper include: (i) a database containing mugshots and corresponding forensic and composite sketches that will be made available to interested researchers; (ii) a comparison of holistic facial representations versus component based representations for sketch to mugshot matching; and (iii) an analysis of the effect of filtering a mugshot gallery using three sources of demographic information (age, gender and race/ethnicity). Our experimental results show that composite sketches are matched with higher accuracy than forensic sketches to the corresponding mugshots. Both of the face representations studied here yield higher sketch to photo matching accuracy compared to a commercial face matcher.

The FaceSketchID System: Matching Facial Composites to Mugshots

“…This publication extends upon the work in [6] in the following ways: (i) the background database is increased to 100,000 versus 10,000 in [6], (ii) integrated and generalized the approaches in [6], [17], [14] and [21] into a unified framework, (iii) improved algorithms such that the recognition performance is comparable to [6] after increasing the background gallery size by an order of magnitude, (iv) evaluated the FaceSketchID System on the largest reported collection of operational (composite, mugshot) pairs reported in the literature, (v) evaluated the effect of the type of training data on the performance of our recognition system, and (vi) demonstrated that composites made from poor quality surveillance imagery can be used to identify the suspect that could not be identified based on the original image.…”

“…However, matching facial composites to photographs is 5 http://www.nist.gov/itl/iad/ig/frvt-home.cfm a more challenging problem with only a limited amount of published work, some of which include: [7], [8], [11], [16], [10], [14], [15], [18], [17], [6]. Of these, most studies have used composites drawn while viewing the mugshot or photograph (viewed hand-drawn composites).…”

“…6. For a given photograph (a) from the AR database [29], the viewed facial composites used in our study consist of a hand-drawn composite (b), a composite created using Identi-Kit (c), and two composites created using FACES (d, e) by two different operators.…”

“…While several methods that match viewed 4 and hand-drawn composites to mugshots have been reported in the literature [7], [8], [11], [16], [10], [15], [14], [18], only a few methods have been published for automatic matching of softwaregenerated composites to mugshots [16], [17]. In all the previous studies reported on software-generated composites, with the exception of [6], composites were created while the operator was viewing the high quality mugshot. This type of viewed composite does not accurately reflect the creation of composites for use in criminal investigations because the mugshot of the suspect is unknown or unavailable.…”

“…Section III describes the construction process for each type of facial composite used in this study. We note that in [6], we referred to hand-drawn composites as "forensic sketches" and software-generated composites as "composite sketches". The naming conventions have been corrected in this study to reflect the prevailing law enforcement terminology.…”