Homogeneity Within Item Forms in Domain Referenced Testing

Abstract

EDUCATIONAL AND PSYCHOLOGICAL MZASURZMZNT 1973, 33, 351-360. IN recent years, criterion-referenced testing has gained considerable attention as an alternative to norm-referenced testing. This is particularly true in &dquo;mastery&dquo; testing where one is interested in gathering information to allow decisions regarding achievement of a defined level of competence, or more generally in the evaluation of instruction.Popham and Husek (1969) have suggested that there are two general kinds of criterion-referenced tests. The first, by far the more common, is composed of items sampled from a population of items which constitute a criterion but for which scores do not convey totally unambiguous information; a score does not tell which items were answered correctly and which were not. The second, which might be considered the ideal, is seldom found. In this case the items are also sampled from the criterion item population but everyone who gets the same score on the test has obtained it in essentially the same manner. Thus, an individual's score implies exactly which items he completed correctly.In dealing with diagnostic tests, it is important to understand the relationships among test items within diagnostic categories and how these relationships compare with an ideal in 1 The authors would like to thank Wells Hively and Harry Patterson for helpful comments about an earlier version of this paper and for providing the data which is discussed in this paper.

Criterion-Referenced Testing and Measurement: A Review of Technical Issues and Developments

“…When items in a domain are expected to be relatively homogeneous, and there are many times when this is not a reasonable assumption (Macready & Merwin, 1973), it has become a fairly common practice for a test developer to compare estimates of item difficulty parameters, or item discrimination parameters, or both. Since one would expect items measuring an objec tive equally well to have similar item parameters, estimates of the parameters are compared to detect items that deviate from the norm.…”

A Latent Trait Look at Pretest-Posttest Validation of Criterion-Referenced Test Items

“…In the preceding paragraphs it was assumed that a single (mastery) continuum underlies the items. Although occasionally the opinion can be met that heterogeneous item domains are usable as well (Millman, 1974), homogeneous item domains are generally considered as a requisite for a criterion-referenced interpretation of test scores (see, e.g., Hambleton & Gorth, 1971;Macready & Merwin, 1973;Popham & Husek, 1967). We denote this underlying mastery continuum by 0 and the cut-off score on it that is used for mastery decisions by the value 0c.…”

A Structural Approach to the Validation of Hierarchical Training Sequences

“…The tasks were too easy. Consequently, it was necessary to select a new set of algebra tasks and psychometrically validate these before proceding to the studies for Task Macready and Merwin (1973) have called a homogeneous item domain.…”