The Interrater Reliability of Elizur's Hostility Systems and Holt's Aggression Variables: A Meta-Analytical Review

Katko, Nicholas J.; Meyer, Gregory J.; Mihura, Joni L.; Bombel, George

doi:10.1080/00223890902936116

Cited by 6 publications

(2 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the Comprehensive System is the dominant approach to the Rorschach, many Rorschach scales were created outside the context of the system. Several reviews have now demonstrated good reliability for nonsystem variables thought to measure aggression (Gacono, Bannatyne-Gacono, Meloy, & Baity, 2005;Katko, Meyer, Mihura, & Bombel, 2009), individuation in relationships , therapy prognosis (Handler & Clemence, 2005), and dependency (Bornstein & Masling, 2005), among others.…”

Section: Reliabilitymentioning

confidence: 99%

The current status of "projective" "tests"

McGrath

Carroll

2012

APA Handbook of Research Methods in Psychology, Vol 1: Foundations, Planning, Measures, and Psychometrics.

View full text Add to dashboard Cite

The term projective tests is often used to encompass a variety of procedures that allow the target individual to provide free-form responses to ambiguous stimuli. The participant's responses are thought to be sensitive to implicit processes, and consequently they may be somewhat resistant to efforts at misrepresentation.This class of instruments has had a particularly checkered past. Because of concerns about honesty in responding to self-report measures, and the psychoanalytic belief that much of mental activity is resistant to self-observation, psychologists became enamored with the potential of projective instruments. The development of the Rorschach Inkblot Method (Rorschach, 1921(Rorschach, /1942 preceded formal discussions of projective psychological tests, but its popularity in the United States is largely attributable to its presumed projective qualities. The Rorschach was soon joined by other instruments, including the Thematic Apperception Test (TAT; Morgan & Murray, 1935;Murray, 1943), the Rosenzweig (1978) Picture-Frustration Study, and the Szondi Test (Deri, 1949). Even tests developed for other purposes came to be used as projectives, particularly the Bender Visual Motor Gestalt Test (Hutt, 1985). A 1959 survey found the three most commonly used psychological tests in clinical practice were projective tests (Sundberg, 1961).By the 1960s, though, the allure was fading for two reasons. One was the general critique of traditional personality assessment that emerged out of behaviorism. Mischel (1968) questioned whether the criterion-related validity of personality measures was sufficient to justify their use, whereas Goldfried and Kent (1972) criticized the practice of using latent constructs to account for associations between test behavior and behavioral outcomes. The second factor was a psychometric critique of projective methods (e.g., Cronbach, 1949;Entwisle, 1972; Swensen, 1957Swensen, , 1968.This second literature has engendered an enduring negative perception of projective instruments in the scientific community. Although surveys in the 1980s and 1990s found more than 75% of clinical doctoral programs required training in projective testing (Piotrowski & Keller, 1984;Piotrowski & Zalewski, 1993), a more recent update saw that rate drop to 59%, with more than half of program directors reporting reduced training in projectives (Belter & Piotrowski, 2001). A recent attempt to generate a list of discredited psychological tests was largely dominated by projective instruments (Norcross, Koocher, & Garofalo, 2006). Only a few instruments, primarily the TAT and Rorschach, continue to appear with regularity in the assessment research literature. The continuing popularity of the former can be traced at least in part to its successful use in motivational research (e.g., McAdams, 1982); that of the latter is directly attributable to the success of Exner's (2003) Comprehensive System, which brought uniformity in administration and scoring, normative data, and interpretation to the Rorschach.In contrast, alt...

show abstract

Section: Reliabilitymentioning

confidence: 99%

The current status of "projective" "tests"

McGrath

Carroll

2012

APA Handbook of Research Methods in Psychology, Vol 1: Foundations, Planning, Measures, and Psychometrics.

View full text Add to dashboard Cite

show abstract

“…IRR has been analyzed and compared for groups with respect to assessee or rater characteristics in journal peer review [ 35 ], grant peer-review [ 5 , 28 , 36 ], classroom observations of teachers [ 37 – 38 ], university candidates [ 3 ], student ratings, etc. In these areas and others, potential exists for assessee covariates, such as gender and ethnicity, rater characteristics such as rater position, experience or training [ 39 – 40 ] or covariates of units, e.g. school type or job type, which may moderate IRR and precision of ratings.…”

Section: Discussionmentioning

confidence: 99%

Disparities in ratings of internal and external applicants: A case for model-based inter-rater reliability

2018

View full text Add to dashboard Cite

Ratings are present in many areas of assessment including peer review of research proposals and journal articles, teacher observations, university admissions and selection of new hires. One feature present in any rating process with multiple raters is that different raters often assign different scores to the same assessee, with the potential for bias and inconsistencies related to rater or assessee covariates. This paper analyzes disparities in ratings of internal and external applicants to teaching positions using applicant data from Spokane Public Schools. We first test for biases in rating while accounting for measures of teacher applicant qualifications and quality. Then, we develop model-based inter-rater reliability (IRR) estimates that allow us to account for various sources of measurement error, the hierarchical structure of the data, and to test whether covariates, such as applicant status, moderate IRR. We find that applicants external to the district receive lower ratings for job applications compared to internal applicants. This gap in ratings remains significant even after including measures of qualifications and quality such as experience, state licensure scores, or estimated teacher value added. With model-based IRR, we further show that consistency between raters is significantly lower when rating external applicants. We conclude the paper by discussing policy implications and possible applications of our model-based IRR estimate for hiring and selection practices in and out of the teacher labor market.

show abstract

A Principal Components Analysis of Rorschach Aggression and Hostility Variables

Katko

Meyer

Mihura

et al. 2010

Journal of Personality Assessment

View full text Add to dashboard Cite

We examined the structure of 9 Rorschach variables related to hostility and aggression (Aggressive Movement, Morbid, Primary Process Aggression, Secondary Process Aggression, Aggressive Content, Aggressive Past, Strong Hostility, Lesser Hostility) in a sample of medical students (N= 225) from the Johns Hopkins Precursors Study (The Johns Hopkins University, 1999). Principal components analysis revealed 2 dimensions accounting for 58% of the total variance. These dimensions extended previous findings for a 2-component model of Rorschach aggressive imagery that had been identified using just 5 or 6 marker variables (Baity & Hilsenroth, 1999; Liebman, Porcerelli, & Abell, 2005). In light of this evidence, we draw an empirical link between the historical research literature and current studies of Rorschach aggression and hostility that helps organize their findings. We also offer suggestions for condensing the array of aggression-related measures to simplify Rorschach aggression scoring.

show abstract

The Interrater Reliability of Elizur's Hostility Systems and Holt's Aggression Variables: A Meta-Analytical Review

Cited by 6 publications

References 37 publications

The current status of "projective" "tests"

The current status of "projective" "tests"

Disparities in ratings of internal and external applicants: A case for model-based inter-rater reliability

A Principal Components Analysis of Rorschach Aggression and Hostility Variables

Contact Info

Product

Resources

About