Evaluator differences in Psychopathy Checklist-Revised factor and facet scores.

Abstract

Recent research suggests that the reliability of some measures used in forensic assessments--such as Hare's (2003) Psychopathy Checklist-Revised (PCL-R)--tends to be weaker when applied in the field, as compared with formal research studies. Specifically, some of the score variability in the field is attributable to evaluators themselves, rather than the offenders they evaluate. We studied evaluator differences in PCL-R scoring among 558 offenders (14 evaluators) and found evidence of large evaluator differences in scoring for each PCL-R factor and facet, even after controlling for offenders' self-reported antisocial traits. There was less evidence of evaluator differences when we limited analyses to the 11 evaluators who reported having completed a PCL-R training workshop. Findings provide indirect but positive support for the benefits of PCL-R training, but also suggest that evaluator differences may be evident to some extent in many field settings, even among trained evaluators.

Competency to stand trial evaluations: A state‐wide review of court‐ordered reports

“…To the extent that evaluator variability in opinions appears excessive in this study, it underscores the concerns about evaluator differences in other samples and other types of evaluation (see, e.g., Boccaccini, Murrie, Rufino, & Gardner, 2014;Boccaccini, Turner, & Murrie, 2008;Harris, Boccaccini, & Murrie, 2015;Murrie & Warren, 2005;Murrie et al, 2008). Indeed, such findings raise important questions about whether and how we can reduce evaluator idiosyncrasy and enhance evaluator reliability in the service of equitable justice (Guarnera, Murrie, & Boccaccini, 2017).…”

“…Indeed, the latter may be the most difficult to remedy, even with training requirements. While training may improve reliability, it may not be sufficient to reduce all evaluator differences or idiosyncrasies; indeed, these appear present, at least to some degree, even among trained clinicians in rigorous research contexts (e.g., Boccaccini et al, ). Furthermore, clinicians may “drift” towards idiosyncratic practice even after adequate training, and are vulnerable to numerous influences that hamper reliability (Guarnera et al, ).…”

Insanity findings and evaluation practices: A state‐wide review of court‐ordered reports

“…Although the results may seem to suggest adversarial allegiance, it is unclear whether these small differences in agreement were actually due to allegiance versus attorney selection effects (see Murrie & Boccaccini, Another perspective on evaluator performance and opinion formationbeyond examining interrater agreement when evaluating the same defendantcomes from examining whether evaluators working within the same system tend to demonstrate the same pattern of findings (e.g., rate of opining insanity) across cases. Recent studies raise questions about evaluator differences in rates of finding criminal defendants incompetent (Murrie, Boccaccini, Zapf, Warren, & Henderson, 2008); finding offenders warrant a paraphilia diagnosis or commitment as a sexually violent predator (Harris, Boccaccini, & Schrantz, 2016); and assigning scores on a popular measure of psychopathic personality (Boccaccini, Murrie, Rufino, & Gardner, 2014;Boccaccini, Turner, & Murrie, 2008). In the only such study of sanity evaluations, researchers considered 59 evaluators who had performed over 10 sanity evaluations each, for a total of 4,498 evaluations, over a 10-year period in Virginia (Murrie & Warren, 2005).…”

Jurors Report that Risk Measure Scores Matter in Sexually Violent Predator Trials, but that Other Factors Matter More

“…Although not designed as a risk measure, the PCL‐R is a 20‐item measure of affective, interpersonal, and behavioral psychopathic traits, scored on the basis of file review and interview, commonly used by experts in SVP evaluations (Jackson & Hess, ). Texas SVP evaluators are required by statute to assess for psychopathy, and all use the PCL‐R (see Boccaccini, Murrie, Rufino, & Gardner, ). Although meta‐analytic findings show that PCL‐R scores are only modest predictors of sexual recidivism ( d = 0.40 for PCL‐R total scores), sexual offenders who score high on the PCL‐R and a measure of sexual deviance are significantly more likely to reoffend (odds ratio = 2.80–3.21) than other offenders (see Hawes, Boccaccini, & Murrie, ).…”