A hierarchy of expert performance.

Abstract

Expert performance can be quantified by examining reliability and biasability between and within experts, and teasing apart their observations from their conclusions. I utilize these parameters to develop a Hierarchy of Expert Performance (HEP) that includes eight distinct levels. Using this hierarchy I evaluate and quantify the performance of forensic experts, a highly specialized domain that plays a critical role in the criminal justice system. Evaluating expert performance within HEP enables the identification of weaknesses in expert performance, and enables the comparison of experts across domains. HEP also provides theoretical and applied insights into expertise.The successful completion of many everyday tasks depends upon human performance abilities; even more so in highly specialized domains, such as aviation, medicine, forensics, finance, and policing, where expert performance is critical. In these and other domains, we trust our money, health, and even our lives to the hands of experts. Most experts are talented people with many years of training and experience and therefore are in high demand and well paid.But, how well do competent experts actually perform? Even defining expertise and who is an expert has been a complex and challenging task with a variety of views and disagreements (Feldon, 2007;Hoffman, 1996). Advances in quantifying expert performance are mostly limited to domain-specific areas of expertise, and hence there is a lack of measurements and a unified framework to assess expert performance across domains.An important component of expert performance across domains relates to biasability and reliability. Biasability refers to the ability to make decisions based on relevant information without being biased by irrelevant contextual information. For example, a forensic expert who is aware of such information (e.g., that the suspect confessed to the crime, that eyewitnesses identified the suspect, or that the detective believes the suspect is guilty) is biased to incorrectly judge that the forensic evidence (e.g., firearms, handwriting, voice, fingerprints, etc.) matches the suspect and can wrongly identify the suspect (Dror & Cole, 2010).In addition to issues of biasability there are basic reliability issues. That is, how reliable (i.e., how consistent, how reproducible) is expert decision making even when there is no exposure to irrelevant biasing information? Would experts ଝ Please note that this paper was handled by the current editorial team of JARMAC.

Cascading Bias of Initial Exposure to Information at the Crime Scene to the Subsequent Evaluation of Skeletal Remains,

“…Previous studies addressing contextual influences and forensic anthropology have shown that when ambiguity is involved in the assessment of skeletal remains, a strong context (such as DNA) influenced the interpretation of participants with regard to sex assessments on the skeletal remains . However, this study highlights that influence can vary and the results shows that the strength of the context in which the decision is being made, as well as the direction of the bias played a significant role.…”

The Forensic Confirmation Bias: A Comparison Between Experts and Novices

“…Dror (27) proposed the hierarchy of expert performance (HEP) framework as an overall single framework that coherently explicates the different aspects of human expert performance, including forensic experts. The framework consists of two main components: biasability and reliability.…”

“…As we found no general measure or scale to define an expert (10,27) we defined experts in the present study as crime scene investigators who have experience in the field. In the Netherlands, experienced crime scene investigators are trained at the Dutch Police Academy.…”

Diagnosing Crime and Diagnosing Disease: Bias Reduction Strategies in the Forensic and Clinical Sciences

“…Who is to make this decision as to which information is relevant, and which is irrelevant or contextual? It is only by deconstructing the process of expert decision‐making that we can fully understand the different sources of cognitive bias.…”