Bayesian Hierarchical Random Effects Models in Forensic Science

Abstract

Statistical modeling of the evaluation of evidence with the use of the likelihood ratio has a long history. It dates from the Dreyfus case at the end of the nineteenth century through the work at Bletchley Park in the Second World War to the present day. The development received a significant boost in 1977 with a seminal work by Dennis Lindley which introduced a Bayesian hierarchical random effects model for the evaluation of evidence with an example of refractive index measurements on fragments of glass. Many models have been developed since then. The methods have now been sufficiently well-developed and have become so widespread that it is timely to try and provide a software package to assist in their implementation. With that in mind, a project (SAILR: Software for the Analysis and Implementation of Likelihood Ratios) was funded by the European Network of Forensic Science Institutes through their Monopoly programme to develop a software package for use by forensic scientists world-wide that would assist in the statistical analysis and implementation of the approach based on likelihood ratios. It is the purpose of this document to provide a short review of a small part of this history. The review also provides a background, or landscape, for the development of some of the models within the SAILR package and references to SAILR as made as appropriate.

Dynamic signatures: A review of dynamic feature variation and forensic methodology

“…The more recent probabilistic approach maintains that uncertainty is present in the inference process and should be handled with statistical tools and models. Recently recommended statistical evidence evaluation approaches rely on personal probability [193][194][195][196], the Bayes theorem [190,195,[197][198][199][200] and to some extent decision theory [201][202][203]. Recommendations from European Institutes and American case law suggest that the Bayesian approach is more coherent and well adapted to forensic science purposes than categorical conclusions or purely technical information [161,190,191,[204][205][206].…”

A probabilistic formalization of contextual bias in forensic analysis: Evidence that examiner bias leads to systemic bias in the criminal justice system

“…We formalize this task by using probabilities and the ratio form of Bayes' rule, as others have done before [9][10] [7]. The analyst's task is comparing the probability that and have a common source given , , and the information , i.e., , to the probability that and have a different source, given the same information, .…”

“…Reporting the value of evidence using the likelihood ratio has gained traction in Europe and internationally [15]. See [7] for a more detailed overview of the use of the likelihood ratio to evaluate evidence. Through the remainder of this article, we will use the likelihood ratio to quantify the value of evidence, but recognize that it is not the only way that the value of evidence is expressed [16].…”

“…We provide a formal framework for how information is updated in cases of cognitive bias by using the ratio form of Bayes' rule. Bayes' rule is a useful tool for this because it mirrors the adversarial system of the courts, and for this reason it has been used elsewhere in forensics to motivate the use of likelihood ratio-based reporting [7] [8]. We hope this formalization can serve as a tool for transparency, and is useful for both laboratories seeking to minimize bias in their analyses and to researchers developing algorithms to augment forensic decision-making tasks.…”

Interpol review of glass and paint evidence 2016-2019

“…Aitken [ 99 ] reviewed Software for the Analysis and Implementation of Likelihood Ratios’ (SAILR) software package used for analysis and implementation of likelihood ratios in forensic science. He reviewed the history, purpose, and background of the program.…”