Exploring Alternative Test Form Linking Designs With Modified Equating Sample Size and Anchor Test Length

Wang, Lin; Qian, Jiahe; Lee, Yi‐Hsuan

doi:10.1002/j.2333-8504.2013.tb02309.x

Cited by 5 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many linking procedures that place both exams onto the same scale. For example, the fixed parameter with score transformation procedure, and the mean-mean or mean-sigma procedure are commonly used linking procedures [22][23][24]. For these assessments it was found that the mean-mean procedure produced the most acceptable transformation between the two scales.…”

Section: Information Functions and Scale Comparisonmentioning

confidence: 99%

Classical test theory and item response theory comparison of the brief electricity and magnetism assessment and the conceptual survey of electricity and magnetism

Eaton

Johnson

Frank

et al. 2019

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

For proper assessment selection understanding the statistical similarities amongst assessments that measure the same, or very similar, topics is imperative. This study seeks to extend the comparative analysis between the brief electricity and magnetism assessment (BEMA) and the conceptual survey of electricity and magnetism (CSEM) presented by Pollock. This is accomplished by using large samples (N BEMA ¼ 5368 and N CSEM ¼ 9905) within classical test theory (CTT) and item response theory (IRT) frameworks. For the IRT comparison, after consideration of the conceptual content addressed in each assessment, it was assumed that each of these assessments are measuring the same student latent ability (θ), specifically a student's ability to do introductory electricity and magnetism. Via a CTT and IRT analysis it was found that both assessments are essentially equal in overall difficulty. Classical item analysis applied to 7 questions used by both assessments revealed that each assessment functions slightly differently internally. The test information curves found from IRT show that the CSEM has superior information compared to the BEMA in estimating student latent abilities for the entire range of typical latent abilities achieved by students on each assessment, θ ≈ −2 to θ ≈ 3. Information in this case is interpreted as how well a student's latent ability was estimated by an assessment as a function of latent ability. When the circuits questions are removed from the BEMA the majority of the information is lost in the θ ≈ 0 to θ ≈ 2 range. This means the circuits questions on the BEMA are information heavy for higher ability scores. So, special considerations should be made as to which assessment a study uses depending on the specific questions a researcher is attempting to answer.

show abstract

Section: Information Functions and Scale Comparisonmentioning

confidence: 99%

Classical test theory and item response theory comparison of the brief electricity and magnetism assessment and the conceptual survey of electricity and magnetism

Eaton

Johnson

Frank

et al. 2019

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

show abstract

“…Both the item parameter transformation and the IRT true score equating phases were accomplished using ICEDOG software (Robin, Holland, & Hemat, ). The statistics of interest in this study were the standard errors of the estimated scaling coefficients A and B in the item parameter transformation (linking) phase and the standard errors of the scale scores. The grouped jackknifing procedure was implemented to estimate the standard errors in both the linking and the equating phases (Qian, ; Wang et al, ) on the entire sample data of a test form and on each jackknife replicate sample. The jackknifed standard errors of the statistics of interest were estimated as follows.Let

\overset{θ}{true}

be the parameter estimated from the whole sample and

{\overset{true^}{θ}}_{(j)}

be the estimate from the j th jackknife replicate sample.…”

Section: Methodsmentioning

confidence: 99%

“…The statistics of interest in this study were the standard errors of the estimated scaling coefficients A and B in the item parameter transformation (linking) phase and the standard errors of the scale scores. The grouped jackknifing procedure was implemented to estimate the standard errors in both the linking and the equating phases (Qian, ; Wang et al, ) on the entire sample data of a test form and on each jackknife replicate sample. The jackknifed standard errors of the statistics of interest were estimated as follows.…”

Section: Methodsmentioning

confidence: 99%

“…In the reported studies of grouped jackknifing applications in educational assessments, instead of forming cluster‐based jackknife groups, a random grouping approach is often employed to form jackknife groups even though the data may have varying degrees of clustering features (Lu, Haberman, Guo, & Liu, ; Wang, Qian, & Lee, ). In sampling, clustering effects are caused by intercluster correlations, and cluster‐based sampling will yield larger variance estimates than those yielded by random sampling (Cochran, , p. 240; Kish, , p. 162).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Grouping Effects on Jackknifed Variance Estimation for Item Response Theory Scaling and Equating With Cluster‐Based Assessment Data

Wang

Qian

Lee

2018

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

Educational assessment data are often collected from a set of test centers across various geographic regions, and therefore the data samples contain clusters. Such cluster‐based data may result in clustering effects in variance estimation. However, in many grouped jackknife variance estimation applications, jackknife groups are often formed by a random grouping method that ignores the cluster structures of the data. In this study, we constructed both random and cluster‐based jackknife groups for data known to have cluster structures and compared the jackknifed standard errors, yielded by two different grouping methods, of item response theory (IRT) scaling coefficient estimates and equated scores. Three independent data samples from an international test of English were used for the study. The cluster‐based jackknife group results showed relatively larger jackknifed standard errors of scaling coefficient estimates and scale scores than the results of the random jackknife groups for all three data samples. For cluster‐based assessment data, the cluster‐based jackknife approach provides a more appropriate way to estimate the standard errors of the parameters of IRT calibration, scaling, and equating analyses.

show abstract

“…According to Lord (1975), IRT methods of equating is useful, especially in increasing the stability of the scales near the extreme values and in reducing drift in equating chains of several test forms. Wang, Qian, and Lee (2013) evaluated the combined effects of reduced equating sample size and shortened anchor test length on item response theory (IRT)-based linking and equating results. The presence of anchor items requires additional considerations because these items play a large role in determining the equating function.…”

Section: Introductionmentioning

confidence: 99%

Bootstrap Equating Errors for the Common-item Nonequivalent Groups Design: A Comparison of Rasch Equating Methods

Zheng¹

2022

BJE

View full text Add to dashboard Cite

The nonequivalent groups with anchor test (NEAT) equating design are traditionally based on using a single anchor to adjust for differences in test difficulty which is critical to equating test forms in most large-scale testing programs. When tests differ somewhat in content and length, methods based on the item response theory (IRT) model leads to greater stability of equating results. The current study compared standard errors, bias, and root mean square errors using four Rasch IRT equating methods for the nonequivalent groups with anchor test design. The sizes of the equating anchor were employed in all four different Rasch equating methods to investigate how different anchor sizes may impact the test accuracy of the tests by conducting a simulation study.

show abstract

Exploring Alternative Test Form Linking Designs With Modified Equating Sample Size and Anchor Test Length

Cited by 5 publications

References 10 publications

Classical test theory and item response theory comparison of the brief electricity and magnetism assessment and the conceptual survey of electricity and magnetism

Classical test theory and item response theory comparison of the brief electricity and magnetism assessment and the conceptual survey of electricity and magnetism

Grouping Effects on Jackknifed Variance Estimation for Item Response Theory Scaling and Equating With Cluster‐Based Assessment Data

Bootstrap Equating Errors for the Common-item Nonequivalent Groups Design: A Comparison of Rasch Equating Methods

Contact Info

Product

Resources

About