Field Reliability of Comprehensive System Scoring in an Adolescent Inpatient Sample

Abstract

The extent to which the Comprehensive System for the Rorschach is reliably scored has been a topic of some controversy. Although several studies have concluded it can be scored reliably in research settings, little is known about its reliability in field settings. This study evaluated the reliability of both response-level codes and protocol-level scores among 84 adolescent psychiatric inpatients in a clinical setting. Rorschachs were originally administered and scored for clinical purposes. Among response codes, 87% demonstrated acceptable reliability(> .60), and most coefficients exceeded .80. Results were similar for protocol-level scores, with only one score demonstrating less than adequate reliability. The findings are consistent with previous evidence, indicating reliable scoring is possible even in field settings.

The Interrater Reliability of Elizur's Hostility Systems and Holt's Aggression Variables: A Meta-Analytical Review

“…These concerns may also be understood in terms of research reliability versus field reliability. Importantly, evidence indicates that reliability tends to be largely consistent across settings at least for CS variables (McGrath et al, 2005;Meyer et al, 2002). As noted in our online supplement examining potential moderators, varying levels of rater training did not markedly influence reliability results; this also provides partial support for the generalizability of these findings.…”

“…Exner's Comprehensive System (CS;Exner, 2003) is the most popular means for administering, scoring, and interpreting the Rorschach. Good reliability (McGrath et al, 2005;Meyer et al, 2002) and reasonable validity have been shown for many CS scores (Hiller, Rosenthal, Bornstein, Berry, & Brunell-Neuleib, 1999;Meyer & Archer, 2001;Meyer & Viglione, 2008;Mihura, 2008;Viglione & Hilsenroth, 2001). The CS Aggressive Movement (AG) score is assigned for aggressive actions taking place in the present.…”

“…Before getting to this point, however, it is worthwhile to determine whether the Elizur (1949) and Holt (1977Holt ( , 2005 scales can be scored with the same degree of consistency as the newer scales. Thorough evidence documenting good to excellent scoring reliability is readily available for AG (McGrath et al, 2005;Meyer et al, 2002;Viglione & Taylor, 2003) and the extended aggression indexes (Gacono et al, 2005). Our goal was to provide similarly thorough evidence for the historically most commonly used scales.…”

Dealing with negative symbolism of destinations with difficult heritage: Analysis of Ugandas image

“…Factor analysis was conducted on ordinal test items constructed by summarising nominal items that were used to measure symbolism and Cronbach-alpha coefficients were used in measuring the internal consistency of measurement items. McGrath et al (2005) argue that coefficients greater than 0.80 are usually desirable, appropriate and achievable in practice. Suhr and Shay (2009) also argue that depending on the purpose of the instrument, the acceptable reliability coefficient (alpha) levels can be as low as 0.60.…”

One More Datum on Rorschach Form Quality

“…In their work, Acklin, McDowell, Verschell, and Chan (2000) suggest kappa was the most adequate statistic to use. During the first few years of the last decade several important works followed: Meyer et al (2002), Taylor (2003), andMcGrath et al (2005). In all of them, the protocol-level and response-level reliability values found for the variable FQu are notably lower than the values for X+% and X−%.…”