Are forensic scientists too risk averse?

Abstract

Fingerprint examiners maintain decision thresholds that represent the amount of evidence required for an identification or exclusion conclusion. As measured by error rate studies (Proc Natl Acad Sci USA. 2011;108(19):7733-8), these decision thresholds currently exhibit a preference for preventing erroneous identification errors at the expense of preventing erroneous exclusion errors. The goal of this study is to measure the decision thresholds for both fingerprint examiners and members of the general public, to determine whether examiners are more risk averse than potential jury members. To externally measure these decision thresholds, subjects manipulated decision criteria in a web-based visualization that reflects the trade-offs between erroneous identification decisions and erroneous exclusion decisions. Data from fingerprint examiners and the general public were compared to determine whether both groups have similar values as expressed by the placement of the decision criteria. The results of this study show that fingerprint examiners are more risk averse than members of the general public, although they align with error rate studies of fingerprint examiners. Demographic data demonstrate those factors that may contribute to differences in decision criterion placement, both between the two groups and between individuals within a group. The experimental methods provide a rich framework for measuring, interpreting, and responding to the values of society as applied to forensic decision-making.

Statistical feature training improves fingerprint-matching accuracy in novices and professional fingerprint examiners

“…Further, fingerprint examiners’ response bias was generally more conservative than novices. This is consistent with previous research demonstrating that forensic science practitioners do typically have a more conservative response style than novices (Mannering et al, 2021 ; Towler et al, 2018 ; although note that accuracy is optimized when response bias is neutral). Domain-specific training did not shift either novices’ or examiners’ response bias, but domain-general training further increased examiners’ conservative response bias (although this did not have any corresponding shift in sensitivity).…”

Validating strength‐of‐support conclusion scales for fingerprint, footwear, and toolmark impressions

“…The sensitivity (d') value for fingerprint examiners across the two participant groups was 2.39, with a standard deviation for the mated pair distribution of 1.48. This difficulty level is consistent with other error rate studies (Ulery, Hicklin [11]; see Mannering, Vogelsang [10]) and thus the task difficulty appears similar to that of casework. Figure 5 illustrates the results of this modeling, and shows the location of different decision criteria for the two scales.…”

“…These were purchased in the same order from Amazon, although we have no control over the batch origin. We constructed a custom 3D-printed jig that was used to produce striated scrapings in heavy-duty aluminum foil (see Figure 1), and while we collected scrapings at 5 different angles (10,20,35,55, and 80 degrees), we judged the 20-and 35-degree angles to be most representative of what might be produced by tools used on metal window and door frames to gain access to property. Thus, we used only scrapings collected using either of these two angles.…”

“…Examiners knew on each trial which scale they would use, and we fit the data using an extension of signal detection theory [9], which provides separate estimates of examiner ability (through an estimate of signal detection’s sensitivity measure) and examiner risk aversion (through an estimate of decision criteria). Response bias can be thought of as how risk‐averse a participant is when making decision: an examiner who makes more Identification conclusions than their colleagues will make more correct decisions but also has an elevated risk erroneous identification outcomes [10]. Results from both the raw data and the signal detection modeling fits demonstrated that examiners used the Identification conclusion less often when the Support for Common Source conclusion was available.…”

The low prevalence effect in fingerprint comparison amongst forensic science trainees and novices

“…Despite trainees’ overall performance advantage over novices, both groups made more false-alarms by misjudging non-matching pairs as matches when match prevalence was high (i.e., 90% of trials), compared to equal prevalence. Importantly, this is the same error that could result in the wrongful conviction of an innocent suspect in a crime [ 15 , 40 ]. This effect was accompanied by fewer ‘misses’ where participants misjudged fewer matching pairs as non-matches when match prevalence was high, compared to equal prevalence.…”