Effect of Repeaters on Score Equating in a Large‐scale Licensure Test

Abstract

This study investigated the subgroup invariance of equating functions for a licensure test in the context of a nonequivalent groups with anchor test (NEAT) design. Examinees who had taken a new, to-be-equated form of the test were divided into three subgroups according to their previous testing experience: (a) repeaters who previously took the reference form, to which the new form would be equated; (b) repeaters who previously took any form other than the reference form; and (c) first-time test takers for whom the new form was the first exposure to the test. Equating functions obtained with two subgroups, all repeaters, and first-time test takers were similar to those obtained with the total group, supporting score equatability of the two forms. However, when the repeater subgroup was divided into two groups based on the particular form examinees took previously, subgroup equating functions substantially differed from the total-group equating function, indicating subgroup dependency of score equating. The results indicate that repeater membership needs to be more clearly specified to assess the impact of repeaters on score equating. Such clarification may be especially necessary for high-stakes licensure tests because repeaters tend to perform more poorly on such tests than first-time test takers.

Current state and future directions of the National Board of Chiropractic Examiners

“…Excluding repeaters helps to control for the possible effect of repeaters on equating. 24,25 If NBCE exams were given on demand to our current number of examinees, we would not have sufficient data to perform psychometric analyses properly, as specified by the best practices detailed in the Standards for Educational and Psychological Testing. 26 Yet, in the context of competency assessment, a computer adaptive testing (CAT) approach may be a conceivable alternative to on-demand tests.…”

Effect of Statistically Matching Equating Samples for Common‐Item Equating

“…Although the difference was small on average, not all conditional differences between the total group conversion and subgroup conversions were negligible, indicating the impact of subgroup composition on the equating. Kim and Walker (2009) investigated the population invariance of equating by comparing equating functions derived using two distinct subgroups (first‐time test taker versus repeater) from the examinee population for a large‐scale licensure test. In their comparisons, equating functions remained essentially invariant across all repeaters versus first‐time test takers, supporting score equitability of the two forms.…”