Kernel Equating Under the Non-Equivalent Groups With Covariates Design

Abstract

When equating two tests, the traditional approach is to use common test takers and/or common items. Here, the idea is to use variables correlated with the test scores (e.g., school grades and other test scores) as a substitute for common items in a non-equivalent groups with covariates (NEC) design. This is performed in the framework of kernel equating and with an extension of the method developed for post-stratification equating in the non-equivalent groups with anchor test design. Real data from a college admissions test were used to illustrate the use of the design. The equated scores from the NEC design were compared with equated scores from the equivalent group (EG) design, that is, equating with no covariates as well as with equated scores when a constructed anchor test was used. The results indicate that the NEC design can produce lower standard errors compared with an EG design. When covariates were used together with an anchor test, the smallest standard errors were obtained over a large range of test scores. The results obtained, that an EG design equating can be improved by adjusting for differences in test score distributions caused by differences in the distribution of covariates, are useful in practice because not all standardized tests have anchor tests.

Linking With External Covariates: Examining Accuracy by Anchor Type, Test Length, Ability Difference, and Sample Size

“…Previous research has shown that a single anchor test can sometimes be insufficient when equating with nonequivalent groups (Moses et al, 2011 Wiberg & Bränberg, 2015). Frequency estimation was nearly always the most accurate method, with the smallest errors tending to come from the NEATC design (similar to Wiberg & Bränberg, 2015, who used kernel methods).…”

“…Otherwise, common variables are expected to produce linking functions with which population invariance may not necessarily hold, as differences in subgroups on external covariates may not reflect differences on the total test. For example, this may be the case in Wiberg and Bränberg (2015), where supplemental information did not come directly from the total test. The term equating is used here unless a distinction between linking and equating is necessary.…”

A Simulation Study to Compare Nonequivalent Groups With Anchor Test Equating and Pseudo‐Equivalent Group Linking

“…One advantage of the PEG method is that it has the power of including as many variables as available. This feature makes PEG distinct from other linking methods that utilize regression methods with limited background variables to link test scores (e.g., Brancerg & Wiberg, ; Wiberg & Branberg, ),…”

“…One advantage of the PEG method is that it has the power of including as many variables as available. This feature makes PEG distinct from other linking methods that utilize regression methods with limited background variables to link test scores (e.g., Brancerg & Wiberg, 2011;Wiberg & Branberg, 2015), For illustration purposes, we chose limited background variables in this study. We intended to evaluate the impact of background variables on the accuracy of PEG linking with respect to their strength in predicting the total scores.…”

Optimal Scores: An Alternative to Parametric Item Response Theory and Sum Scores

“…Where deemed appropriate, covariates are often available that shed further light on the state of the test taker. The use of covariates such as age, gender, educational background, and so on is not encouraged in academic testing, although there are recent examples of using covariates successfully in test equating(Wiberg and Bränberg, 2015). Covariates are certainly important in clinical assessment contexts provided that disclosure issues and other considerations are properly dealt with.…”