Linking With External Covariates: Examining Accuracy by Anchor Type, Test Length, Ability Difference, and Sample Size

Albano, Anthony D.; Wiberg, Marie

doi:10.1177/0146621618824855

Cited by 7 publications

(8 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the precision of linking estimates can be improved by including further person covariates in the analysis (Albano and Wiberg 2019;Sansivieri and Wiberg 2019). This could be particularly true if there also exist DIF effects for person covariates.…”

Section: Discussionmentioning

confidence: 99%

Robust and Nonrobust Linking of Two Groups for the Rasch Model with Balanced and Unbalanced Random DIF: A Comparative Simulation Study and the Simultaneous Assessment of Standard Errors and Linking Errors with Resampling Techniques

Robitzsch¹

2021

Preprint

View full text Add to dashboard Cite

In this article, the Rasch model is used for assessing a mean difference between two groups for a test of dichotomous items. It is assumed that random differential item functioning (DIF) exists that has the potential to bias group differences. The case of balanced DIF is distinguished from the case of unbalanced DIF. In balanced DIF, DIF effects cancel out on average. In contrast, in unbalanced DIF, the expected value of DIF effects can differ from zero and favors a particular group on average. Robust linking methods (e.g., invariance alignment) aim at determining group mean differences that are robust to the presence of DIF. In contrast, group differences obtained from nonrobust linking methods (e.g., Haebara linking) can be affected by the presence of a few DIF effects. Alternative robust and nonrobust linking methods are compared in a simulation study under various simulation conditions. It turned out that robust linking methods are preferred over nonrobust alternatives in the case of unbalanced DIF effects. Moreover, M-estimation theory is used for studying the asymptotic behavior of linking estimators if the number of items tends to infinity. These results give insights into asymptotic bias and the estimation of linking errors that represent the variability in estimates due to selecting items in a test. Moreover, M-estimation theory is also used in an analytical treatment to assess standard errors and linking errors simultaneously. Finally, double jackknife and double half sampling methods are introduced and evaluated in a simulation study to assess standard errors and linking errors simultaneously. Half sampling outperformed jackknife estimators for the assessment of variability of estimates from robust linking methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Robust and Nonrobust Linking of Two Groups for the Rasch Model with Balanced and Unbalanced Random DIF: A Comparative Simulation Study and the Simultaneous Assessment of Standard Errors and Linking Errors with Resampling Techniques

Robitzsch¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The results obtained in their study indicated that even if the composition of the group taking the exam changes, test results can be controlled. In the study conducted by Albano and Wiberg (2019), in which gender was used as a common variable, it was determined that frequency estimation gives less error in the presence of anchor test and covariate variables. Moreover, recent studies comparing NEAT and NEC designs show that common variables can be used instead of common items.…”

Section: Studies Comparing Neat and Nec Designmentioning

confidence: 99%

“…However, specifically in examinations that are applied several times in a year or term, using the same anchor test sometimes can cause some problems; for example, the use of the same items repeatedly can lead to recall of items for individuals, which can negatively affect discrimination. Recently, as a solution to this problem, there are studies suggesting that test equating can be conducted by using nonequivalent groups with covariates (NEC) design (e.g., Akın Arıkan, 2020; Albano & Wiberg, 2019;Branberg, 2010;Branberg & Wiberg, 2011;Gonzales et al, 2015;Wiberg & Branberg, 2015;Wiberg & von Davier, 2017). For example, Yurtçu (2018) equated scores obtained from different tests by using common item scores, gender, and mathematics self-efficacy scores as covariates.…”

Section: Introductionmentioning

confidence: 99%

Comparison of Kernel equating methods under NEAT and NEC designs

ÖZSOY¹,

Kilmen

2023

International Journal of Assessment Tools in Education

View full text Add to dashboard Cite

In this study, Kernel test equating methods were compared under NEAT and NEC designs. In NEAT design, Kernel post-stratification and chain equating methods taking into account optimal and large bandwidths were compared. In the NEC design, gender and/or computer/tablet use was considered as a covariate, and Kernel test equating methods were performed by using these covariates and considering bandwidths. The study shows that, in the NEAT design, Kernel chain equating methods exhibit higher error than the post-stratification equating methods do since the lowest error in the NEC design was obtained from the Kernel equating method with large bandwidth through the computer/tablet variable. Kernel test equating results based on the NEC design, which considers gender and computer tablet use variables as a covariate separately, showed lower SEE than that of the NEC pattern, which takes these variables together as covariates. In terms of the bandwidth, when all methods are compared within the pattern used (i.e., NEAT and NEC), it has been seen that generally Kernel test equating with large bandwidth results in fewer errors than the Kernel test equating with optimal bandwidth. When the NEAT and NEC designs are compared generally, the NEAT design has a lower SEE than that of the NEC design.

show abstract

“…When neither single group nor random groups designs are feasible, it may be possible to use a set of common items (sometimes called an anchor test ) as a form‐invariant connective. This design, denoted common‐item nonequivalent groups design , is widely used in large‐scale testing (Albano & Wiberg, 2019; Kolen & Brennan, 2013). Here, scores on the anchor test are considered comparable regardless of the form on which they appear.…”

Section: Connectivesmentioning

confidence: 99%

Historical Perspectives on Score Comparability Issues Raised by Innovations in Testing

Baldwin

Clauser

2022

J Educational Measurement

View full text Add to dashboard Cite

While score comparability across test forms typically relies on common (or randomly equivalent) examinees or items, innovations in item formats, test delivery, and efforts to extend the range of score interpretation may require a special data collection before examinees or items can be used in this way-or may be incompatible with common examinee or item designs altogether. When comparisons are necessary under these nonroutine conditions, forms still must be connected by something and this article focuses on these form-invariant connective somethings. A conceptual framework for thinking about the problem of score comparability in this way is given followed by a description of three classes of connectives. Examples from the history of innovations in testing are given for each class.

show abstract

Linking With External Covariates: Examining Accuracy by Anchor Type, Test Length, Ability Difference, and Sample Size

Cited by 7 publications

References 29 publications

Robust and Nonrobust Linking of Two Groups for the Rasch Model with Balanced and Unbalanced Random DIF: A Comparative Simulation Study and the Simultaneous Assessment of Standard Errors and Linking Errors with Resampling Techniques

Robust and Nonrobust Linking of Two Groups for the Rasch Model with Balanced and Unbalanced Random DIF: A Comparative Simulation Study and the Simultaneous Assessment of Standard Errors and Linking Errors with Resampling Techniques

Comparison of Kernel equating methods under NEAT and NEC designs

Historical Perspectives on Score Comparability Issues Raised by Innovations in Testing

Contact Info

Product

Resources

About