The expression and interpretation of uncertain forensic science evidence: Verbal equivalence, evidence strength, and the weak evidence effect.

Abstract

Standards published by the Association of Forensic Science Providers (2009, Standards for the formulation of evaluative forensic science expert opinion, Science & Justice, Vol. 49, pp. 161-164) encourage forensic scientists to express their conclusions in the form of a likelihood ratio (LR), in which the value of the evidence is conveyed verbally or numerically. In this article, we report two experiments (using undergraduates and Mechanical Turk recruits) designed to investigate how much decision makers change their beliefs when presented with evidence in the form of verbal or numeric LRs. In Experiment 1 (N = 494), participants read a summary of a larceny trial containing inculpatory expert testimony in which evidence strength (low, moderate, high) and presentation method (verbal, numerical) varied. In Experiment 2 (N = 411), participants read the same larceny trial, this time including either exculpatory or inculpatory expert evidence that varied in strength (low, high) and presentation method (verbal, numerical). Both studies found a reasonable degree of correspondence in observed belief change resulting from verbal and numeric formats. However, belief change was considerably smaller than Bayesian calculations would predict. In addition, participants presented with evidence weakly supporting guilt tended to "invert" the evidence, thereby counterintuitively reducing their belief in the guilt of the accused. This "weak evidence effect" was most apparent in the verbal presentation conditions of both experiments, but only when the evidence was inculpatory. These findings raise questions about the interpretability of LRs by jurors and appear to support an expectancy-based account of the weak evidence effect.

Understanding forensic expert evaluative evidence: A study of the perception of verbal expressions of the strength of evidence

“…The study did not compare responses to those from an alternative scale, but instead measured the extent of any divergence from a model in which intentions and perceptions align. Following previous work [5,6,7,8], this study provides additional evidence that the potential for miscommunication when verbal expressions are used and the perception problems observed suggest a number of divergences between the intended meaning of the expressions and the way in which they are perceived. The results and findings generated by this study highlight the need for the undertaking of further validation studies in alternative contexts.…”

“…In an investigation into the expression and interpretation of the verbal scale, Martire et al [7] undertook experiments that revealed evidence of a "weak evidence effect" whereby some participants inverted the direction of support when presented with evidence that provided weak support. Martire et al [8] explored different methods of communicating support.…”

Lay understanding of forensic statistics: Evaluation of random match probabilities, likelihood ratios, and verbal equivalents.

“…Research has generally found that simulated jurors are less responsive to forensic evidence than Bayesian models indicate they should be (Thompson & Schumann, 1987;Faigman & Baglioni, 1988;Goodman, 1992;Smith et al, 1996;Schklar & Diamond, 1999;Nance & Morris, 2002, 2005Martire et al, 2013Martire et al, , 2014; for reviews of the early studies see Koehler, 2001;Kaye & Koehler, 1991;Thompson, 1989). But Thompson, Kaasa, and Peterson (2013) recently questioned whether jurors always underutilize forensic evidence relative to Bayesian norms.…”

“…This finding supports hypothesis 4 -that judgments of the chances of guilt made using the odds measure would be less responsive to the forensic evidence (lesser shifts in judgments) than judgments made using the log scale. Martire et al (2013Martire et al ( , 2014 reported that people sometimes give lower estimates of the odds of guilt after receiving forensic evidence that is characterized as weakly supportive of the prosecution (a finding they call the "weak evidence effect"). Our experiment had no conditions in which an expert characterized the forensic evidence as "weak," but we nevertheless found that a small percentage of our participants (7.6% for the Log scale; 6.4% for the Odds measure) judged the defendant less likely to be guilty after receiving the forensic evidence than before-in other words, they treated the forensic evidence as exculpatory.…”

“…It recommends that analysts say the evidence provides "weak or limited" support for the favored hypothesis when the likelihood ratio is 1-10; "moderate support" when the likelihood ratio is 10 -100; "moderately strong" support when the likelihood ratio is 100 -1,000; "strong support" when it is 1,000 -10,000; "very strong" support when it is 10,000 to 1 million; and "extremely strong" support when it is over 1 million. But recent research has raised questions about whether these labels convey the intended meaning to jurors (Martire et al, 2013;Martire, Kemp, Sayle, & Newell, 2014).…”