A second look at the validity of widely used Rorschach indices: Comment on Mihura, Meyer, Dumitrascu, and Bombel (2013).

Abstract

We comment on the meta-analysis by Mihura, Meyer, Dumitrascu, and Bombel (2013), which examined the validity of scores in Exner's Comprehensive System (CS) for the Rorschach. First, we agree there is compelling evidence that 4 categories of cognitive scores-the "Rorschach cognitive quartet"-are related to cognitive ability/impairment and thought disorder. We now feel comfortable endorsing the use of these scores in some applied and research settings. Second, we conducted new meta-analyses (k = 44) for the 4 noncognitive Rorschach scores with highest validity in the Mihura et al. findings. Unlike Mihura et al., we included unpublished dissertations (although we did not attempt to exhaustively unearth all unpublished studies), calculated correlations instead of semipartial correlations, and used the Rorschach International Norms for a larger proportion of comparisons. Our validity estimates for the Suicide Constellation and Weighted Sum of Color were similar to or even higher than those of Mihura et al., although we concluded that support for the Suicide Constellation is limited and that Weighted Sum of Color probably does not measure its intended target. Our validity estimates for Sum Shading and the Anatomy and X-ray score were much lower than those of Mihura et al. We conclude that their meta-analysis accurately reflects the published literature, but their exclusion of unpublished studies led to substantial overestimates of validity for some and perhaps many Rorschach scores. Therefore, the evidence is presently insufficient to justify using the CS to measure noncognitive characteristics such as emotionality, negative affect, and bodily preoccupations.

Standards, accuracy, and questions of bias in Rorschach meta-analyses: Reply to Wood, Garb, Nezworski, Lilienfeld, and Duke (2015).

“…

Wood, Garb, Nezworski, Lilienfeld, and Duke (2015) found our systematic review and meta-analyses of 65 Rorschach variables to be accurate and unbiased, and hence removed their previous recommendation for a moratorium on the applied use of the Rorschach. However, Wood et al (2015) hypothesized that publication bias would exist for 4 Rorschach variables.

…”

“…However, Wood et al (2015) hypothesized that publication bias would exist for 4 Rorschach variables. To test this hypothesis, they replicated our meta-analyses for these 4 variables and added unpublished dissertations to the pool of articles.…”

Form Quality in Rorschach Comprehensive System and R-PAS: Sample of Psychiatric Cases

“…Consequently, those Rorschach indexes that include FQ play an important role not only in the clinical setting, in which they can support diagnosis and help to understand the distress of the individual and on the prognosis and better treatment planning, but also in the research context. In fact, several recent studies have tested the psychometric properties of these indexes (Mihura, Meyer, Dumitrascu, & Bombel, 2013;Viglione, Giromini, Gustafson, & Meyer, 2014;Wood, Garb, Nezworski, Lilienfeld, & Duke, 2015), and applied them on several samples from different countries (DzamonjaIgnjatovic, Smith, Djuric Jocic, & Milanovic, 2013;Moore, Viglione, Rosenfarb, Patterson, & Mausbach, 2013;Silva & Costa, 2014). Valkonen, Lindfors, and Knekt (2012) used the Ego Impairment Index, EII, to study anxiety disorders, and found a significant, but weak association between the EII and the interview-based scale Level of Personality Organization, LPO, (Pyykkönen, 2008, apud Valkonen et al, 2012.…”

“…Villemor-Amaral, Yazigi, Nascimento, Primi, and Semer (2007) examined location areas and their respective FQ of 370 Brazilian non-patient adult subjects with a view to the construction of a Brazilian Atlas of specific Localization Areas with their respective FQ list. The validity studies on FQ in the CS in Brazil were: Marques, Chaves, and Yazigi (2012) who examined psychiatric patients and found that the XA% variable, which combines FQ+, FQo and FQu, was sensitive to discriminate perceptual acuity in patients with psychotic features and non-psychotic patients, using both Exner's FQ listing (Exner, 1995(Exner, /1999 and the Brazilian FQ listing (Villemor-Amaral et al, 2007).…”

“…The validity studies on FQ in the CS in Brazil were: Marques, Chaves, and Yazigi (2012) who examined psychiatric patients and found that the XA% variable, which combines FQ+, FQo and FQu, was sensitive to discriminate perceptual acuity in patients with psychotic features and non-psychotic patients, using both Exner's FQ listing (Exner, 1995(Exner, /1999 and the Brazilian FQ listing (Villemor-Amaral et al, 2007). Pianowski and VillemorAmaral (2010) coded protocols on the FQ using both Exner (1995Exner ( /1999 and the Brazilian FQ listing (Villemor-Amaral et al, 2007) in a non-patient sample, and found that the Brazilian tables resulted in higher means for FQo, XA% and X+%, and a lower mean for FQ-. Leonel, Semer, and Yazigi (2012) have also tested the FQ Brazilian list in a non-patient sample and found that FQo% and X+% have higher means, while FQu, FQ-, XA%, Xu% and X-% had lower means when using the Brazilian list.…”