A Comparison of IRT Observed Score Kernel Equating and Several Equating Methods

Abstract

Item response theory (IRT) observed score kernel equating was evaluated and compared with equipercentile equating, IRT observed score equating, and kernel equating methods by varying the sample size and test length. Considering that IRT data simulation might unequally favor IRT equating methods, pseudo tests and pseudo groups were also constructed to make equating results comparable with those from the IRT data simulation. Identity equating and the large sample single group rule were both set as criterion equating (or true equating) on which local and global indices were based. Results show that in random equivalent groups design, IRT observed score kernel equating is more accurate and stable than others. In non-equivalent groups with anchor test design, IRT observed score equating shows lowest systematic and random errors among equating methods. Those errors decrease as a shorter test and a larger sample are used in equating; nevertheless, effect of the latter one is ignorable. No clear preference for data simulation method is found, though still affecting equating results. Preferences for true equating are spotted in random Equivalent Groups design. Finally, recommendations and further improvements are discussed.

Comparison of Kernel Equating and Kernel Local Equating in Item Response Theory Observed Score Equating

“…As the number of anchor items increased, the errors of the estimates in both methods were closer to one another. Wang et al (2020) also obtained similar results where the number of items was 30 and 45 in the simulation. This finding is not supported by the findings of Uysal (2014), in which he found that error estimates with the Stocking-Lord method were lower than the Haebara method.…”

“…Öztürk-Gübeş (2019), on the other hand, investigated the effect of multidimensionality on test equating and not included the change in the item numbers. Moreover, Wang et al (2020) compared equapercentile equating, Kernel equating, and IRT Kernel equating methods.…”

Conversion Between the Mini-Mental State Examination and the Montreal Cognitive Assessment for Patients With Different Forms of Dementia

“…One advantage of this method is that equivalent scores are always within an interval of possible achievable scores, even though irregular score distributions are possible. 21 Therefore, to ensure a normal distribution of the test scores, log-linear smoothing of each measure's raw scores was applied leading to higher accuracy. Training and validation samples show similar results, therefore supporting our analysis.…”

Equating Two Health-related Quality of Life Instruments Applied to Cancer Survivors Who Underwent Palliative Care

“…By selecting common items across two HRQOL instruments, scores of a single instrument can be compatible with the scores of the other instrument (i.e., test equating). While traditional equating methods are available on test equating, equating methods using IRT model are known for more accurate and stable [24,25,34,35]. The equating across the instruments may be a promising method.…”