Validation of a coupled, multiple response assessment for upper-division thermal physics

Rainey, Katherine D.; Vignal, Michael; Wilcox, Bethany R.

doi:10.1103/physrevphyseducres.18.020116

Cited by 7 publications

(1 citation statement)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The development and validation of the U-STEP using Classical Test Theory are reported in Refs. [9,13,14]. An exploratory factor analysis of data from the U-STEP indicated that the U-STEP's factor structure is dominated by a single factor, and thus, the instrument appears statistically unidimensional.…”

Section: Context and Methodsmentioning

confidence: 99%

Methods for utilizing Item response theory with Coupled, Multiple-Response assessments

Wilcox¹,

Rainey²,

Vignal³

2022

2022 Physics Education Research Conference Proceedings

Self Cite

View full text Add to dashboard Cite

Recent years have seen a movement within the research-based assessment development community towards item formats that go beyond simple multiple-choice formats. Some have moved towards free-response questions, particularly at the upper-division level; however, free-response items have the constraint that they must be scored by hand. To avoid this limitation, some assessment developers have moved toward formats that maintain the closed-response format, while still providing more nuanced insight into student reasoning. One such format is known as coupled, multiple response (CMR). This format pairs multiple-choice and multiple-response formats to allow students to both commit to an answer in addition to selecting options that correspond with their reasoning. In addition to being machine-scorable, this format allows for more nuanced scoring than simple right or wrong. However, such nuanced scoring presents a potential challenge with respect to utilizing certain testing theories to construct validity arguments for the assessment. In particular, Item Response Theory (IRT) models often assume dichotomously scored items. While polytomous IRT models do exist, each brings with it certain constraints and limitations. Here, we will explore multiple IRT models and scoring schema using data from an existing CMR test, with the goal of providing guidance and insight for possible methods for simultaneously leveraging the affordances of both the CMR format and IRT models in the context of constructing validity arguments for research-based assessments.

show abstract

Section: Context and Methodsmentioning

confidence: 99%