The Nature of Objectivity With the Rasch Model

Abstract

Although it has been claimed that the Rasch model leads to a higher degree of objectivity in measurement than has been previously possible, this model has had little impact on test development. Population-invariant item and ability calibrations, together with the statistical equivalency of any two item subsets, are supposedly possible if the item pool has been calibrated by the Rasch model. Initial research has been encouraging, but the implications of underlying assumptions and operational computations in the Rasch model for trait theory have not been clear from previous work. The current paper presents an analysis of the conditions under which the claims of objectivity will be substantiated, with special emphasis on the nature of equivalent forms. It is concluded that the real advantages of the Rasch model will not be apparent until the technology of trait measurement becomes more sophisticated.A procedure for a new kind of item analysis, based on the Rasch (1960, 1961, 1966a, 1966b) logistic model, is now available for use in developing measures of unidimensional traits. Wright (1967), one of the first researchers to operationalize the Rasch model, claims that the use of this model leads to an objectivity in measurement which is not possible under classical approaches to test development. The objectivity, according to Rasch (1961), results from two basic features of the model: 1) the calibration of the test items is independent of the sample and 2) the measurement of a person on the latent trait is independent of the particular items used.A psychological test having these general characteristics would become directly analogous to a yardstick that measures the length of objects. That is, the intervals on the yardstick are independent of the length of the objects and the length of individual objects is interpretable without respect to which particular yardstick is used. In contrast, tests developed according to the classical model have neither characteristic. The score obtained by a person is not interpretable without referring to both some norm group and the particular test form used. Wright and Panchapakesan (1969) claim that objective measurement is now possible because their estimation techniques for the Rasch parameters yield tests with the following specific properties: 1)the estimates of the item difficulty parameter will not vary significantly over different samples of people, 2) the estimates of a person's ability, given a certain raw score, will be invariant over different samples, and 3) estimates of a person's ability from any calibrated subset of items will be statistically equivalent. If these properties are truly now possible through application of the Rasch model, it would seem that mental measurement would be revolutionized. No longer would equivalent forms need to be carefully developed, since measurement is instrument independent and any two subsets of the calibrated item pool could be used as alternative mNow at

Applicability of the Rasch Model with Varying Item Discriminations

“…The obtained sample had a mean of 0.0 and a sample variance of 1.475. While it is known that the best item calibration is done with a good deal of replicability within each score group, hence large N (Whitely & Dawis, 1974), the computer time and cost were prohibitive for this. The number 75 was considered sufficient because (1) an additional calibration was obtained on an &dquo;infinite&dquo; sample, and (2) the 75 &dquo;persons&dquo; used were &dquo;centered on the test&dquo; (i.e., the test was of exactly the right difficulty for them), resulting in a very efficient administration with respect to amount of information obtained.…”

“…In 1960, Rasch (cited in Lord & Novick, 1968;Whitely & Dawis, 1974;Wright & Panchapakesan, 1969) presented three models to explain misreadings, number of words read, and general achievement. Each of these is a twoparameter model, incorporating only the ability of each person and the difficulty of each test item to explain the observed data.…”

A Cross-Cultural Analysis of the Fairness of the Cattell Culture Fair Intelligence Test Using the Rasch Model

“…Since estimates of item parameters now would not vary significantly unless the item differed in &dquo;cultural&dquo; loadings (Whitely & Dawis, 1974), any significant differences in item parameter estimates would indicate cultural bias in that item.…”

Statistical Models for Criterion-Referenced Testing and Decisionmaking

“…that an individual with ability A will pass item i with easiness parameter E . The Rasch probability function (Whitely and Davis, 1974, p.1 6 4 ) is…”

“…Thus, the probabilities a person will pass the various individual items must remain invariant, regardless if the ability test contains the whole item pool or only some subset of items. (Whitely and Davis, 1974, p.1 6 5 )…”