A Tutorial on Mechanical Decision-Making for Personnel and Educational Selection

Meijer, Rob R.; Neumann, Marvin; Hemker, Bas T.; Niessen, A. Susan M.

doi:10.3389/fpsyg.2019.03002

Cited by 16 publications

(22 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Yet, results from (some relatively old) surveys suggest that mechanical combination is rarely used in psychological practice (Ryan et al, 2015;Ryan & Sackett, 1987;Vrieze & Grove, 2009), which is also referred to as "algorithm aversion" (Dietvorst et al, 2015, p. 114). Algorithm aversion is problematic 1 , because it results in suboptimal and untransparent judgments and decisions, which hinders the evaluation and improvement of the decision process (Meijer et al, 2020). Therefore, researchers called for (qualitative) investigations of why decision makers underutilize algorithms (Burton et al, 2020;Dietvorst et al, 2015;.…”

Section: Holistic and Mechanical Combination In Psychological Assessm...mentioning

confidence: 99%

Holistic and mechanical combination in psychological assessment: Why algorithms are underutilized and what is needed to increase their use

Neumann¹,

Niessen²,

Hurks³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Although mechanical combination results in more valid judgments and decisions than holistic combination, existing publications suggest that mechanical combination is rarely used in practice. Yet, these publications are either descriptions of anecdotal experiences or outdated surveys. Therefore, in several Western countries, we conducted two surveys (total N = 323) and two focus groups to investigate (1) how decision makers in psychological and HR practice combine information, (2) why they do (not) use mechanical combination, and (3) what may be needed to increase its use in practice. Many participants reported mostly using holistic combination, usually in teams. The most common reasons for not using mechanical combination were that algorithms are unavailable in practice and that stakeholders do not accept their use. Furthermore, decision makers do not quantify information, do not believe in research findings on evidence-based decision making, and think that combining holistic and mechanical combination results in the best decisions. The most important reason why mechanical combination is used was to increase predictive validity. To stimulate the use of mechanical combination in practice, our results suggest that decision makers should receive more training on evidence-based decision making, and decision aids supporting the use of mechanical combination should be developed.

show abstract

Section: Holistic and Mechanical Combination In Psychological Assessm...mentioning

confidence: 99%

Holistic and mechanical combination in psychological assessment: Why algorithms are underutilized and what is needed to increase their use

Neumann¹,

Niessen²,

Hurks³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This may, in turn, help researchers and practitioners demystify coaches' mental modeling and nudge coaches to monitor, reference, and track the efficacy of their criteria over time. As demonstrated in fields such as medicine and education, this adoption and reliance on established criteria can mitigate the influence of bias in the judgement and decision-making process, and in turn, improve accuracy (Arkes et al, 2006 ; Meijer et al, 2020 ).…”

Section: Selection Criteria Policies – the Good The Bad And The Riskymentioning

confidence: 99%

Storm Clouds on the Horizon: On the Emerging Need to Tighten Selection Policies

Johnston

Farah

Baker

2021

Front. Sports Act. Living

View full text Add to dashboard Cite

Athlete selection is fundamental in elite sport, occurring regularly throughout an athlete's development. Research in this area reveals the accuracy of these decisions is questionable in even the most elite sport environments and athletes are increasingly disputing these decisions as unfair and punitive. As a countermeasure to these dispute and arbitration practices, many elite sport systems have created policies where coaches must outline and stand behind the criteria used for their selection decisions. Selection criteria policies have the potential to help encourage fair selection practices by holding selectors accountable to their selection criteria, but their implementation also has the potential to wrongfully nudge selectors toward developing more defendable, but less-accurate selection practices. The paper concludes with 10 suggestions to help support practitioners when implementing selection criteria.

show abstract

“…The innovation proposed here is an application of a SPJ decision model to MSO data and forensic opinions in an effective, user-friendly procedure. In contrast to fears that a structured prediction model is rigid, cumbersome, or overly technical (which have been identified as sources of clinician resistance to structured methods; Vrieze and Grove, 2009 ; Lilienfeld et al, 2013 ), the proposed model focuses on the reduction of data collection to the most powerful predictors central to the legal standard using a simple worksheet format (Meijer et al, 2020 ; see Figure 2 ).…”

Section: An Spj Decision Model For Criminal Responsibilitymentioning

confidence: 99%

Improving Criminal Responsibility Determinations Using Structured Professional Judgment

Acklin

Velasquez

2021

Front. Psychol.

View full text Add to dashboard Cite

Forensic psychologists commonly utilize unstructured clinical judgment in aggregating clinical and forensic information in forming opinions. Unstructured clinical judgment is prone to evaluator bias and suboptimal levels of inter-rater reliability. This article proposes Structured Professional Judgment (SPJ) methods as a potential remedy. Following a review of canonical forensic assessment models, the prevalence of bias in forensic judgments, and inter-rater agreement in criminal responsibility (CR) determinations, this article presents a SPJ model for CR evaluations translated from violence risk assessment methodology. A systematic user-friendly methodology is described, applying procedural checklists, application of a mental state at time of the offense (MSO) model using structured data collection methods, aggregation of empirical evidence guidelines, and post-hoc hypothesis testing using the Analysis of Competing Hypotheses (ACH). A case study describes application of the procedural and CR decision model in a complex homicide case. The model demonstrates the power and efficacy of the application of SPJ to forensic decision-making and is relevant to other types of forensic assessment (e.g., competency to stand trial, post-acquittal release decision-making).

show abstract

A Tutorial on Mechanical Decision-Making for Personnel and Educational Selection

Cited by 16 publications

References 39 publications

Holistic and mechanical combination in psychological assessment: Why algorithms are underutilized and what is needed to increase their use

Holistic and mechanical combination in psychological assessment: Why algorithms are underutilized and what is needed to increase their use

Storm Clouds on the Horizon: On the Emerging Need to Tighten Selection Policies

Improving Criminal Responsibility Determinations Using Structured Professional Judgment

Contact Info

Product

Resources

About