Testlet Response Theory and Its Applications

Wainer, Howard; Bradlow, Eric T.; Wang, Xiaohui

doi:10.1017/cbo9780511618765

Cited by 283 publications

(326 citation statements)

References 0 publications

Supporting

Mentioning

314

Contrasting

Unclassified

Order By: Relevance

“…Such items are defined as testlet by Wainer and Kiely (1987). Testlets defined to be longer than a single item but shorter than a whole test (Wainer, Bradlow, & Wang, 2007). There are studies in the related literature indicating that such items ruin the local independency assumption and are locally dependent (DeMars, 2006;Lee, 2004;Rijmen, 2010;Thissen, Steinberg, & Mooney, 1989;Zhang, 2010).…”

Section: Introductionmentioning

confidence: 99%

Examination of Different Item Response Theory Models on Tests Composed of Testlets

Koğar¹,

Kelecioğlu²

2017

JEL

View full text Add to dashboard Cite

The purpose of this research is to first estimate the item and ability parameters and the standard error values related to those parameters obtained from Unidimensional Item Response Theory (UIRT), bifactor (BIF) and Testlet Response Theory models (TRT) in the tests including testlets, when the number of testlets, number of independent items, and sample size change, and then to compare the obtained results. Mathematic test in PISA 2012 was employed as the data collection tool, and 36 items were used to constitute six different data sets containing different numbers of testlets and independent items. Subsequently, from these constituted data sets, three different sample sizes of 250, 500 and 1000 persons were selected randomly. When the findings of the research were examined, it was determined that, generally the lowest mean error values were those obtained from UIRT, and TRT yielded a mean of error estimation lower than that of BIF. It was found that, under all conditions, models which take into consideration the local dependency have provided a better model-data compatibility than UIRT, generally there is no meaningful difference between BIF and TRT, and both models can be used for those data sets. It can be said that when there is a meaningful difference between those two models, generally BIF yields a better result. In addition, it has been determined that, in each sample size and data set, item and ability parameters and correlations of errors of the parameters are generally high.

show abstract

Section: Introductionmentioning

confidence: 99%

Examination of Different Item Response Theory Models on Tests Composed of Testlets

Koğar¹,

Kelecioğlu²

2017

JEL

View full text Add to dashboard Cite

show abstract

“…Because the observables within a task all depend on the same stimulus material, and often on a common response, one would expect that the observables might not be conditionally independent given the relevant proficiency variables. Introducing an additional latent variable to account for task-specific dependence (similar to Wainer, Bradlow, & Wang, 2007) seems like a viable approach; Almond, Mulder, Hemat and Yan (2006) discuss some possible ways that this could be implemented with Bayesian networks. However, when this simulation study was expanded to include these extra latent variables and parameters, getting the MCMC algorithm to converge was problematic, suggesting that the extra latent variables introduce multiple modes or large flat spaces in the likelihood surface.…”

Section: Discussionmentioning

confidence: 99%

Parameter Recovery Studies With a Diagnostic Bayesian Network Model

2008

View full text Add to dashboard Cite

This paper describes a Bayesian network model for a candidate assessment design that had four proficiency variables and 48 tasks with 3-12 observable outcome variables per task and scale anchors to identify the location of the subscales. The domain experts' view of the relationship among proficiencies and tasks established a complex prior distribution over 585 parameters. Markov Chain Monte Carlo (MCMC) estimation recovered the parameters of data simulated from the expert model. The sample size and the strength of the prior had only a modest effect on parameter recovery, but did affect the standard error of estimated parameters. Finally, an identifiability issue involving relabeling of proficiency states and permutations of the matrixes is addressed in the context of this study. Scope of StudiesBayesian networks form an attractive class of models for diagnostic assessments as they can relate multiple proficiency variables to multiple dependent observables coming from 1. There were multiple correlated proficiencies, which must be simultaneously estimated. 2. The observable outcome variables were grouped into tasks of varying complexity; each task tapped one or more proficiencies. 3. Tasks had different target difficulties. 4. The tasks were to be presented in complex balanced incomplete block design to ensure sufficient concentration around each of seven potential proficiencies.This paper describes simulation studies designed to answer the question of "Can an assessment with these characteristics be modeled with a Bayesian network?" assuming that there was no local dependence between observables in the same task. This paper describes a number of parameter recovery experiments based on the assessment design described in Section 2. The methodology was roughly similar for all of the experiments. In particular,Key Words and Phrases: Bayesian networks, model construction, Markov Chain Monte Carlo (MCMC) estimation, prior elicitation, scale anchor * Educational Testing Service, MS 13-E Princeton, NJ 08541 USA Mail Address: ralmond@ets.org, almond@acm.org 1) Later versions of this assessment were named iSkills TM . Note that the work described here is based on a very preliminary model of the ICT Literacy which has undergone considerable refinement since the studies described here were carried out and that the models described in this paper do not reflect current thinking about this construct or the scoring methods used in the iSkills assessment.

show abstract

“…Some possible future extensions are: the inclusion of multidimensional IRT models (Reckase 2009); cognitive diagnosis CAT models (Cheng 2009;Kaplan, de la Torre, and Barrada 2015); new or other methods for item exposure and content balancing control; and testlet IRT models (Wainer, Bradlow, and Wang 2009).…”

Section: Final Commentsmentioning

confidence: 99%

Computerized Adaptive Testing with R: Recent Updates of the Package catR

Magis

Barrada²

2017

J. Stat. Soft.

View full text Add to dashboard Cite

The purpose of this paper is to list the recent updates of the R package catR. This package allows for generating response patterns under a computerized adaptive testing (CAT) framework with underlying item response theory (IRT) models. Among the most important updates, well-known polytomous IRT models are now supported by catR; several item selection rules have been added; and it is now possible to perform post-hoc simulations. Some functions were also rewritten or withdrawn to improve the usefulness and performances of the package.

show abstract

Testlet Response Theory and Its Applications

Cited by 283 publications

References 0 publications

Examination of Different Item Response Theory Models on Tests Composed of Testlets

Examination of Different Item Response Theory Models on Tests Composed of Testlets

Parameter Recovery Studies With a Diagnostic Bayesian Network Model

Computerized Adaptive Testing with R: Recent Updates of the Package catR

Contact Info

Product

Resources

About