Effect of calibration data on forensic likelihood ratio from a face recognition system

Abstract

A biometric system used for forensic evaluation requires a conversion of the score to a likelihood ratio. A likelihood ratio can be computed as the ratio of the probability of a score given the prosecution hypothesis is true and the probability of a score given the defense hypothesis is true. In this paper we study two different approaches of a forensic likelihood ratio computation in the context of forensic face recognition. These approaches differ in the databases they use to obtain the score distribution under the prosecution and the defense hypothesis and therefore consider slightly different interpretation of these hypotheses. The goal of this study is to quantify the effect of these approaches on the resultant likelihood ratio in the context of evidence evaluation from a face recognition system. A state-of-the art commercial face recognition system is employed for facial images comparison and computation of scores. A simple forensic case is simulated by randomly selecting a small subset from the FRGC database. Images in this subset are used to estimate the score distribution under the prosecution and the defense hypothesis and the effect of different approaches of a likelihood ratio computation is demonstrated and explained. It is observed that there is a significant variation in the resultant likelihood ratios given the databases which are used to model the prosecution and defense hypothesis are varied.

The logical foundations of forensic science: towards reliable knowledge

“…More recently, mathematical methods of calibration have been developed and extended to provide evaluative approaches to facial comparison [19] and to speech comparison [20,21]. There is much scope for applying these methods to other areas where the comparison process is complex, such as the examples given in the previous paragraph.…”

Avoiding overstating the strength of forensic evidence: Shrunk likelihood ratios/Bayes factors

“…Score-based approaches are increasingly popular across multiple branches of forensic science, e.g., [17][18][19][20][21][22][23][24][25]. Quantitative measurements made on objects of interest such as voice recordings, face images, and glass fragments usually result in multivariate data with complex distributions.…”

Biometric score calibration for forensic face recognition

“…An alternate approach is to compare pairs of biometric specimens where each pair is obtained from a same-source [6,15]. This eliminates the need of the suspect's biometric specimens in computation of the training scores in the s p set.…”

“…The third alternative approach is to use pairs where the two the biometric specimens are from two different persons in the relevent potential population. [3,6,10,15]. Please refer to [4] for an overview of the biometric data sets collection in forensic casework for a LR computation.…”

“…An important condition in forensic LR computation is that the pairs of biometric specimens used for training should reflect the conditions of the pair of biometric specimens for which the LR is computed. Variability exists in the selection of the different-sources and 3.3 Quantification of the sampling variability in forensic likelihood-ratio computation from biometric scores 41 same-source pairs of biometric specimens for training, e.g., either the trace or the trace-like suspect's biometric specimen can be compared to the biometric specimens of the potential population (possible potential sources of the trace biometric specimen) to compute the different-sources scores [3,6,10,15]. Same-source scores can be obtained by comparing trace-like biometric specimens from the suspect to the reference biometric specimens of the suspect or using a general approach, where pairs of biometric specimens from a samesource are compared.…”