Likelihood Ratio as Weight of Forensic Evidence: A Closer Look

Abstract

The forensic science community has increasingly sought quantitative methods for conveying the weight of evidence. Experts from many forensic laboratories summarize their fndings in terms of a likelihood ratio. Several proponents of this approach have argued that Bayesian reasoning proves it to be normative. We fnd this likelihood ratio paradigm to be unsupported by arguments of Bayesian decision theory, which applies only to personal decision making and not to the transfer of information from an expert to a separate decision maker. We further argue that decision theory does not exempt the presentation of a likelihood ratio from uncertainty characterization, which is required to assess the ftness for purpose of any transferred quantity. We propose the concept of a lattice of assumptions leading to an uncertainty pyramid as a framework for assessing the uncertainty in an evaluation of a likelihood ratio. We demonstrate the use of these concepts with illustrative examples regarding the refractive index of glass and automated comparison scores for fngerprints.Key words: assumptions lattice; Bayes' factor; Bayes' rule; Bayesian decision theory; subjective probability; uncertainty; uncertainty pyramid. Executive SummaryIn response to calls from the broader scientifc community [1,2] and concerns of the general public, experts in many disciplines of forensic science have increasingly sought to develop and use objective or quantitative methods to convey the meaning of evidence to others, such as an attorney or members of a jury. Support is growing, especially in Europe [3,4], for a recommendation that forensic experts communicate their fndings using a "likelihood ratio" (see Appendix A for an introduction to likelihood ratios). Proponents of this approach [5 11] appear to believe that it is supported by Bayesian reasoning, a paradigm often viewed as normative (i.e., the right way; what someone should use) for making decisions when uncertainty exists [12 14].Individuals following Bayesian reasoning may establish their personal degrees of belief regarding the truth of a claim in the form of odds (i.e., ratio of their probability that the claim is true to their probability that the claim is false), taking into account all information currently available to them. Upon encountering new evidence, individuals quantify their "weight of evidence" as a personal likelihood ratio. Following Bayes' rule, individuals multiply their previous (or prior) odds by their respective likelihood ratios to obtain their updated (or posterior) odds, refecting their revised degrees of belief regarding the claim in question. Because the likelihood ratio is subjective and personal, we fnd that the proposed framework in which a forensic expert provides a likelihood ratio for others to use in Bayes' equation is unsupported by Bayesian Journal of Research of National Institute of Standards and Technology decision theory, which applies only to personal decision making and not to the transfer of information from an expert to a separate decision maker, s...

Differential retention of pollen grains on clothing and the effectiveness of laboratory retrieval methods in forensic settings

“…However, although this takes into account unequal abundance of different species, it does not account for potential differential retention of the species on the fabrics. One the strengths of a Bayesian approach is the ability to weight different lines of evidence in a model (NRC, 2009;Lund and Iyer, 2017). In theory therefore, it would be possible to use general trends relating to pollen or fabric characteristics, as well as important taphonomic variables (such as pollen productivity) to improve the efficiency of the model in characterising the strength of the evidence.…”

Wetting and spreading of human blood: Recent advances and applications

“…The impor- 30 tance of understanding this problem for forensic science and biomedical applications, appears to be crucial since it is used as a tool to obtain evidence in crime scene reconstitution or in medical diagnosis. The National Institute of Standards and Technology (NIST) has pointed out, in a 35 very recent report, the urge of using valid scientific methods before presenting evidence in courtrooms [22]. This illustrates the demand that exists concerning this research.…”

Unreliable evidence in binary classification problems

“…Bayesian classification also is used in the evaluation of forensic evidence, particularly through likelihood ratios. National Institute of Standards and Technology (NIST) statisticians recently looked at the question of uncertainty characterization for this context [18].…”