Repeatability and Reproducibility of Decisions by Latent Fingerprint Examiners

Abstract

The interpretation of forensic fingerprint evidence relies on the expertise of latent print examiners. We tested latent print examiners on the extent to which they reached consistent decisions. This study assessed intra-examiner repeatability by retesting 72 examiners on comparisons of latent and exemplar fingerprints, after an interval of approximately seven months; each examiner was reassigned 25 image pairs for comparison, out of total pool of 744 image pairs. We compare these repeatability results with reproducibility (inter-examiner) results derived from our previous study. Examiners repeated 89.1% of their individualization decisions, and 90.1% of their exclusion decisions; most of the changed decisions resulted in inconclusive decisions. Repeatability of comparison decisions (individualization, exclusion, inconclusive) was 90.0% for mated pairs, and 85.9% for nonmated pairs. Repeatability and reproducibility were notably lower for comparisons assessed by the examiners as “difficult” than for “easy” or “moderate” comparisons, indicating that examiners' assessments of difficulty may be useful for quality assurance. No false positive errors were repeated (n = 4); 30% of false negative errors were repeated. One percent of latent value decisions were completely reversed (no value even for exclusion vs. of value for individualization). Most of the inter- and intra-examiner variability concerned whether the examiners considered the information available to be sufficient to reach a conclusion; this variability was concentrated on specific image pairs such that repeatability and reproducibility were very high on some comparisons and very low on others. Much of the variability appears to be due to making categorical decisions in borderline cases.

Fingerprint identification: advances since the 2009 National Research Council report

“…Most changes of opinion were towards inconclusive decisions. However, none of the four false positive errors included in this study were repeated [36]. The observed lack of repeatability and reproducibility increases with the difficulty or complexity of the mark as judged by the examiners.…”

“…The accuracy, reliability, repeatability and reproducibility of fingerprint decisions have been tested in the so-called 'black box' study [35,36]. It details, for a corpus of experts, the percentages of decisions of identification and exclusion that indeed correspond to ground truth.…”

LFIQ: Latent fingerprint image quality

“…Reliable latent quality assessment is crucial, primarily to prevent any evidence of value from being discarded. Recent studies [20,21] and evaluations of latent matchers [13] show that latent value determination by examiners can be subjective and inconsistent, which motivates us to develop an objective measure of latent quality. We have proposed a latent quality metric, called LFIQ, that incorporates ridge quality, minutiae reliability, and finger position in latent image.…”

“…same latent print [20,21], and (iii) heavy workload faced by the examiners in latent print units can lead to insufficient amount of time for analysis which can result in false decisions on latent value [15].…”

Fingerprint image analysis: role of orientation patch and ridge structure dictionaries

“…Hence, latent examiners and AFIS work collaboratively in a framework called Analysis, Comparison, Evaluation and Verification (ACE-V) (Ashbaugh 1999). However, human involvement in latent examination has raised some concerns related to repeatability and reliability (Ulery et al 2011(Ulery et al , 2012. Furthermore, when the comparison time (between latent and exemplar prints) is limited, latent examiners are more likely to make an inconclusive matching decision (Dror et al 2011).…”