The most <i>consistent</i> finding in forensic science is <i>inconsistency</i>

Dror, Itiel E.

doi:10.1111/1556-4029.15369

Cited by 12 publications

(4 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, this process lacks consistency. The lack of consistency is a major problem in forensic medicine, both from a scientific and a criminal justice point of view [45] (Figure 5). On the other hand, progress in computing programming has ignited diligence for learning the AI-constructed machines built for the high-dimensional output of data [57].…”

Section: Discussionmentioning

confidence: 99%

Postmortem Gone Astray—A Systematic Review and Meta-Analysis

Šoša,

Perković

2024

Forensic Sciences

View full text Add to dashboard Cite

Premortem clinical diagnoses and postmortem autopsy findings do not match historically. These clinicopathological discrepancies are expected to be higher during the coronavirus disease 2019 (COVID-19) pandemic due to increased pressure on healthcare. At the same time, autopsies and clinicopathological discrepancies accurately display the quality of the healthcare system. To assess that, we examined the relevant works of literature according to the PRISMA-based protocol. As a discrepancy rate change-related antemortem scrutiny of medical care in the hospital, we also checked whether studies with patients from the intensive care unit (ICU) differed in discrepancies significantly. We found similar overall risk differences in “pre-COVID” and “during the pandemic” groups of studies. Based on this, we concluded that healthcare quality did not drop significantly during the pandemic. Be that as it may, the pandemic exposed some shortcomings in mortem healthcare regarding consensus to the autopsy, organ retention, burial, and the postponed burial or the digitalization of postmortem healthcare. All of these issues should be addressed in the future.

show abstract

Section: Discussionmentioning

confidence: 99%

Postmortem Gone Astray—A Systematic Review and Meta-Analysis

Šoša,

Perković

2024

Forensic Sciences

View full text Add to dashboard Cite

show abstract

“…Expert judgment is indispensable, yet infamously inconsistent. Decades of research in areas as different as agriculture, forensics, economics, and medicine support this conclusion (Dror, 2023;Litvinova et al, 2022). In health professions education, experts often function as examiners in rater-based assessment.…”

Section: Introductionmentioning

confidence: 99%

“…Current thinking on why between-expert inconsistencies emerge (Dror, 2023) as well as findings from research on human decision making (Dror, 2020;Gigerenzer et al, 2022) might help to address this issue. Clearly, assessment in health professions education often involves decision making.…”

Section: Introductionmentioning

confidence: 99%

Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions

Schauber,

Olsen,

Werner

et al. 2024

Adv in Health Sci Educ

View full text Add to dashboard Cite

Introduction Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that inconsistencies in ratings depend on the level of performance a to-be-evaluated candidate shows. This possibility has not been addressed deliberately and with appropriate statistical methods. By adopting the theoretical lens of ecological rationality, we evaluate if easily implementable strategies can enhance decision making in real-world assessment contexts. Methods We address two objectives. First, we investigate the dependence of rater-consistency on performance levels. We recorded videos of mock-exams and had examiners (N=10) evaluate four students’ performances and compare inconsistencies in performance ratings between examiner-pairs using a bootstrapping procedure. Our second objective is to provide an approach that aids decision making by implementing simple heuristics. Results We found that discrepancies were largely a function of the level of performance the candidates showed. Lower performances were rated more inconsistently than excellent performances. Furthermore, our analyses indicated that the use of simple heuristics might improve decisions in examiner pairs. Discussion Inconsistencies in performance judgments continue to be a matter of concern, and we provide empirical evidence for them to be related to candidate performance. We discuss implications for research and the advantages of adopting the perspective of ecological rationality. We point to directions both for further research and for development of assessment practices.

show abstract

“…Recently Dror [1] raised the subject of inconsistency in forensic science. He stated: “The most consistent finding in many forensic science domains is inconsistency (i.e., lack of reliability, reproducibility, repeatability, and replicability).…”

Section: Introductionmentioning

confidence: 99%

Extending the discussion on inconsistency in forensic decisions and results

Buckleton,

Bright,

Taylor

et al. 2024

Journal of Forensic Sciences

View full text Add to dashboard Cite

The subject of inter‐ and intra‐laboratory inconsistency was recently raised in a commentary by Itiel Dror. We re‐visit an inter‐laboratory trial, with which some of the authors of this current discussion were associated, to diagnose the causes of any differences in the likelihood ratios (LRs) assigned using probabilistic genotyping software. Some of the variation was due to different decisions that would be made on a case‐by‐case basis, some due to laboratory policy and would hence differ between laboratories, and the final and smallest part was the run‐to‐run difference caused by the Monte Carlo aspect of the software used. However, the net variation in LRs was considerable. We believe that most laboratories will self‐diagnose the cause of their difference from the majority answer and in some, but not all instances will take corrective action. An inter‐laboratory exercise consisting of raw data files for relatively straightforward mixtures, such as two mixtures of three or four persons, would allow laboratories to calibrate their procedures and findings.

show abstract

The most consistent finding in forensic science is inconsistency

Cited by 12 publications

References 45 publications

Postmortem Gone Astray—A Systematic Review and Meta-Analysis

Postmortem Gone Astray—A Systematic Review and Meta-Analysis

Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions

Extending the discussion on inconsistency in forensic decisions and results

Contact Info

Product

Resources

About