Commentary on: Thompson WC. Uncertainty in probabilistic genotyping of low template DNA: a case study comparing STRmix™ and TrueAllele™. J Forensic Sci. 2023;68 (3):1049–63. doi: 10.1111/1556‐4029.15225

Kalafut, Tim; Curran, James M.; Coble, Michael D.; Buckleton, John

doi:10.1111/1556-4029.15405

Cited by 3 publications

(14 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem is compounded rather than solved by the new evidence that Buckleton et al present in their comment [2]. They report that the results produced by STRMix itself vary markedly as the analytic threshold is changed.…”

Section: Buckleton Et Al Candidly Acknowledge the Difficulty Of Dealingmentioning

confidence: 99%

“…It suggested specific language that might be used in a revision. I was pleased that Buckleton et al acknowledged that this may be a suitable matter for SWGDAM to consider [2]. OSAC should also look at this issue.…”

Section: Cons Ervative Rep Orting Of E Xculpatory E Viden Cementioning

confidence: 99%

“…If different analytic thresholds produce very different LRs, and we cannot tell which of these LRs is more trustworthy, then how should we evaluate the strength of the evidence in such a case? Buckleton et al express "frustration with comparison studies," like my article [2]. Drawing analogy with weather forecasting, they suggest there is little value in knowing that two forecasters differ if we do not know which is better calibrated.…”

Section: Buckleton Et Al Candidly Acknowledge the Difficulty Of Dealingmentioning

confidence: 99%

“…with low template DNA: "Most of us are wary of low peak heights. This feeling of discomfort is developed from a large body of experience noting the pernicious effects of artifacts …" [2]. When discussing whether it makes sense to lower analytic thresholds in order to include more alleles in the PG analysis, they acknowledge that the behavior of PG systems in such circumstances "is based entirely on whether correct or incorrect peak information is added to the inputs" and acknowledge that "[w]hether the change as the AT is lowered is beneficial or not cannot be learnt from the LR" [2].…”

Section: Buckleton Et Al Candidly Acknowledge the Difficulty Of Dealingmentioning

confidence: 99%

“…This feeling of discomfort is developed from a large body of experience noting the pernicious effects of artifacts …" [2]. When discussing whether it makes sense to lower analytic thresholds in order to include more alleles in the PG analysis, they acknowledge that the behavior of PG systems in such circumstances "is based entirely on whether correct or incorrect peak information is added to the inputs" and acknowledge that "[w]hether the change as the AT is lowered is beneficial or not cannot be learnt from the LR" [2]. They quote with apparent approval a statement in my article [1] that: "Whether lowering the threshold increases the accuracy of the program by allowing it better to distinguish low-level contributors or reduces accuracy by causing it to draw inferences from unreliable data can best be determined by testing accuracy with known samples of the type in question.…”

Section: Buckleton Et Al Candidly Acknowledge the Difficulty Of Dealingmentioning

confidence: 99%

See 4 more Smart Citations

Author's response

Thompson

2023

Journal of Forensic Sciences

View full text Add to dashboard Cite

See Original Article here See Commentary on here Editor, When I learned that two programs for probabilistic genotyping had produced widely different results for the same case, I wanted to understand how that could have happened. My article [1] about the case was a first cut at explaining an event that I found extraordinary. In their comment, Buckleton et al. [2] suggest that there is really nothing surprising about what I observed. By their account, the two programs are similar and produce different outputs only because they received different inputs due to differing analytic thresholds. To the forensic science community, they seem to be saying: "move along folks, there is nothing to see here." They appear to agree with some of my points about reporting PG results, but they suggest that the attention I devoted to comparing the performance of STRMix and TrueAllele in this case "was not warranted" [2].Yet there is far more going on here than "different inputs producing different outputs." It is worth repeating that the two programs were analyzing the same case using the same data file. While the analytic thresholds the analysts chose to apply were different, there is no way to know which threshold was more appropriate, and hence no way of knowing which of the widely different result is more trustworthy. The suggestion that the programs actually led to the same "interpretation" because both produced exculpatory results is unconvincing when the likelihood ratios (LRs) supporting that interpretation differed by five to six orders of magnitude. While DNA evidence ultimately played a minor role in the case that I discussed, DNA evidence of a similar type-based on "very low information content profiles"-is playing an increasingly important role in the legal system. Hence, the conflicting results and approaches to analysis and reporting that occurred in this case will certainly re-occur in other cases. Indeed, in a non-peer-reviewed online posting, Mark Perlin and his employees assert that "Cybergenetics has re-analyzed STRMix 'inconclusive' or weak LR results on 85 evidence items, finding stronger exclusionary results for 55 of them (65%)" [3]. While Perlin et al. [3] make a number of false assertions in the same online posting [4], their comments make it clear that the case I highlighted is not the only one in which the two PG systems have produced differing results. In my view, such differences warrant careful consideration by the scientific and legal communities.

show abstract

Section: Buckleton Et Al Candidly Acknowledge the Difficulty Of Dealingmentioning

confidence: 99%

Section: Cons Ervative Rep Orting Of E Xculpatory E Viden Cementioning

confidence: 99%

Section: Buckleton Et Al Candidly Acknowledge the Difficulty Of Dealingmentioning

confidence: 99%

Section: Buckleton Et Al Candidly Acknowledge the Difficulty Of Dealingmentioning

confidence: 99%