How reliable are Psychopathy Checklist–Revised scores in Canadian criminal trials? A case law review.

Abstract

The Psychopathy Checklist-Revised (PCL-R;Hare, 2003) is a professional rating scale that enjoys widespread use in forensic and correctional settings, primarily as a tool to inform risk assessments in a variety of types of cases (e.g., parole determinations, sexually violent predator [SVP] civil commitment). Although widely described as "reliable and valid" in research reports, several recent field studies have suggested that PCL-R scores provided by examiners in forensic cases are significantly less reliable than the interrater reliability values reported in research studies. Most of these field studies, however, have had small samples and only examined SVP civil commitment cases. This study builds on existing research by examining the reliability of PCL-R scores provided by forensic examiners in a much more extensive sample of Canadian criminal cases. Using the LexisNexis database, we identified 102 cases in which at least 2 scores were reported (of 257 total PCL-R scores). The single-rater intraclass correlation coefficient (ICCA1) was .59, indicating that a large percentage of the variance in individual scores was attributable to some form of error. ICC values were somewhat higher for sexual offending cases (.66) than they were for nonsexual offending cases (.46), indicating that poor interrater reliability was not restricted specifically to the assessment of sexual offenders. These and earlier findings concerning field reliability in legal cases suggest that the standard error of measurement for PCL-R scores that are provided to the courts is likely to be much larger than the value of 2.90 reported in the instrument's manual.

An update and expansion on the role of the Violence Risk Appraisal Guide and Historical Clinical Risk Management‐20 in United States case law

“…More frequently, however, the written decision provided little context and explanation for how, if at all, the court factored this testimony into the final opinion, a finding consistent with previous case law reviews (e.g., DeMatteo & Edens, 2006;Edens et al, 2015). We echo the suggestion of DeMatteo et al (2014) that a comprehensive review of trial transcripts from various geographic regions may provide additional, more robust, data as to how these measures are used in criminal proceedings and how much attention is given to the measures and mental health testimony by the court.…”

“…This coding form was modeled after forms used in previous case law reviews (DeMatteo et al, ; Edens et al, ) while including items pertinent to the present research question. Coding was completed by two doctoral‐level psychologists and three doctoral graduate students.…”

“…Further, although surveys of forensic clinicians provide data as to what measures clinicians used during the evaluation process, they do not provide information as to how these measures are understood or considered by legal decision‐makers. Since these earlier surveys, researchers have turned to case law reviews to add to this research and better understand how psychological instruments are reported in case law (e.g., DeMatteo & Edens, ; DeMatteo et al, ; Edens, Cox, Smith, DeMatteo, & Sorman, ; Vitacco et al, ). Although not without limitations, case law reviews provide insight into how violence risk measures are employed by forensic evaluators as well as how they are used by legal decision‐makers.…”

Dangerous, depraved, and death‐worthy: A meta‐analysis of the correlates of perceived psychopathy in jury simulation studies

“…Perhaps foremost, a growing body of field literature (see Edens & Boccaccini, ) suggests that scores from this instrument are not particularly reliable in applied contexts in which mental health experts are introducing them. Although the professional manual (Hare, ) provides intraclass correlation (ICC) statistics across various offender samples in the 0.85 and above range, numerous field studies have suggested that it may be closer to 0.60 (or even lower) in adversarial forensic settings (e.g., Edens, Cox, Smith, DeMatteo, & Sörman, ; Miller, Kimonis, Otto, Kline, & Wasserman, ). Of particular concern, the aforementioned Factor 1 traits seem to be even less reliable, producing field results in which sometimes more than half the variance in scores is the result of some form of error (Miller et al, ).…”

Lay Theories and Attitudes About Psychopathy

“…Mental health professionals frequently use psychopathy in their assessments when assessing risk of violence (Hurducas, Singh, de Ruitner, & Petrila, 2014;Singh, Bjørkly, & Fazel, 2016;Viljoen, McLachlan, & Vincent, 2010). Psychopathy is often used in a variety of court cases (e.g., parole determinations, sexually violent predator civil commitments) and particularly in violent cases where a dangerous offender label is in question Edens, Cox, Smith, DeMatteo, & Sörman, 2015).…”