Illusions of Meaning in the Ipsative Assessment of Children's Ability

Abstract

In this study, we analyze the relative efficacy of normative and ipsative measures for the study of intra-and interindividual differences in child ability. With the use of representative data sets, including the WISC-R national standardization sample, purely ipsatized (or deviational ipsative) subtest scores were contrasted with conventional norm-based scores in terms of the evidential and consequential bases for validity. Internal and external evidence for validity was assessed for relative convergence of ability attributes, short-and Paul A. McDermott long-term stability, and predictive efficiency. Comparative utility of each type of measure was explored for theoretical relevance, applicability in measurement work, and assessment of individualized intervention outcomes. Ipsative ability measures were found to be uniformly inferior to their normative counterparts, with ipsative measures conveying no uniquely useful information and otherwise impeding the versatility of assessment.

Increasing the reliability of ipsative interpretations in neuropsychology: A comparison of reliable components analysis and other factor analytic methods

“…Several competing explanations exist for the lack of evidence supporting the clinical utility of difference scores, including over-reliance on null-hypothesis significance testing at the expense of clinically useful statistics such as sensitivity and specificity (Ivnik et al, 2000) 1 , poor predictive validity of difference scores, and0or inadequate reliability of difference scores for making clinical interpretations (Macmann & Barnett, 1997;McDermott et al, 1992;Nunnally & Bernstein, 1994;Streiner & Norman, 1995). The first explanation suggests that difference scores have adequate reliability and validity but that the necessary data have not been reported, whereas the latter two accounts propose that reliability and0or validity may be adequate for group data but not for making decisions about individuals.…”

Temporal Stability of WISC-III Subtest Composite: Strengths and Weaknesses.

“…Stability of cognitive strengths and weaknesses over time is one type of evidence that could support ipsative subtest interpretation. This temporal stability hypothesis was examined with the Wechsler Intelligence Scale for Children-Revised (WISC-R; Wechsler, 1974) with 303 randomly selected children tested twice as part of WISC-R validation studies and with 189 children twice tested for special education eligibility (McDermott, Fantuzzo, Glutting, Watkins, & Baggaley, 1992). Classificatory stability of relative cognitive strengths and weaknesses identified by subtest elevations and depressions was near chance levels for both groups of children.…”

Comparing longitudinal profile patterns of Mathematics and Reading in early child longitudinal study, kindergarten: The Profile Analysis via Multidimensional Scaling (PAMS) approach

“…One commonly used profile analysis is to employ ipsatized scores where the mean is subtracted from each subtest score, yielding a ''profile'' of an individual's strengths and weaknesses in the subtests. According to McDermott and his colleagues, using ipsatized scores on cognitive ability tests is ineffective to differentiate between clinical groups and unsuccessful in predicting academic achievement (McDermott et al 1992;McDermott and Glutting 1997) because the differentiating indices mainly rely on general or overall scores that do not indicate strengths or weaknesses in subscale profiles. Moreover, ipsatized scores do not provide information about how individual's observed score profiles are similar or dissimilar relative to a more representative profile.…”

“…Moreover, ipsatized scores do not provide information about how individual's observed score profiles are similar or dissimilar relative to a more representative profile. Therefore, researchers have turned their attention to identifying major (or core) profiles (McDermott et al 1992), which represent a smaller number of normative profiles reflecting the most typical profile information in a population.…”