Computerized Adaptive Testing Using the Partial Credit Model: Effects Of Item Pool Characteristics and Different Stopping Rules

Dodd, Barbara G.; Koch, William R.; Ayala, R. J. De

doi:10.1177/0013164493053001005

Cited by 32 publications

(40 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Item pools with 30 to 60 items have been found adequate for successful CAT procedures with polytomous items (Dodd, Koch, & De Ayala, 1989; Dodd, Koch, & De Ayala, 1993). For the current study, two real pools of 28 and 95 items, respectively, served as models to generate small (30) and large (90) item pools.…”

Section: Methodsmentioning

confidence: 99%

“…Results indicated that the SE stopping rule yielded higher correlations between known and estimated abilities, while also producing considerably fewer floor/ceiling cases. Additionally, Dodd, Koch, and De Ayala (1993) found that, for adaptive test scored with the partial credit model, the SE stopping rule produced fewer floor/ceiling cases, while also resulting in shorter tests with greater measurement precision than the minimum information stopping rule.…”

mentioning

confidence: 98%

“…Two types of variable length stopping rules have been used (Dodd, Koch, & De Ayala, 1993). These are the standard error (SE) stopping rule and the minimum information stopping rule.…”

mentioning

confidence: 99%

See 2 more Smart Citations

A New Stopping Rule for Computerized Adaptive Testing

Choi

Grady

Dodd

2010

Educational and Psychological Measurement

Self Cite

View full text Add to dashboard Cite

The goal of the current study was to introduce a new stopping rule for computerized adaptive testing. The predicted standard error reduction stopping rule (PSER) uses the predictive posterior variance to determine the reduction in standard error that would result from the administration of additional items. The performance of the PSER was compared to that of the minimum standard error stopping rule and a modified version of the minimum information stopping rule in a series of simulated adaptive tests, drawn from a number of item pools. Results indicate that the PSER makes efficient use of CAT item pools, administering fewer items when predictive gains in information are small and increasing measurement precision when information is abundant. KeywordsComputerized Adaptive Testing (CAT); Stopping Rule; Testing Burden According to Wainer (2000) an adaptive test can be considered complete after a predetermined number of items have been administered, when a predetermined level of measurement precision has been reached, or when a predetermined length of time has elapsed. The two most commonly used methods for determining when a computerized adaptive test is complete are the fixed length and variable length stopping rules.Under a fixed length stopping rule, an adaptive test is terminated when a predetermined number of items have been administered. Accordingly, all examinees are administered the same number of items, regardless of the degree of measurement precision achieved upon termination of the test. The primary advantage of the fixed length stopping rule is its simplicity. However, one consequence of the implementation of the fixed length stopping rule is that examinees will be measured with different degrees of precision, with larger measurement error typically occurring at extreme trait levels. Additionally, a fixed length stopping rule may limit the efficiency of an adaptive test through the unnecessary administration of items that contribute little information about examinee trait level.In contrast, variable length stopping rules typically seek to achieve a certain degree of measurement precision for all examinees, even when doing so means that some examinees are given more items than others. Two types of variable length stopping rules have been Send correspondence to: Seung W. Choi, s-choi@northwestern.edu, Northwestern University Feinberg School of Medicine, Department of Medical Social Sciences, 710 N. Lake Shore Dr., Chicago, IL 60611, USA, Phone: (312) Fax: (312) 503-9800. NIH Public Access Author ManuscriptEduc Psychol Meas. Author manuscript; available in PMC 2011 January 27. NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript used (Dodd, Koch, & De Ayala, 1993). These are the standard error (SE) stopping rule and the minimum information stopping rule. Of these, the most commonly used has been the SE stopping rule, which terminates an adaptive test when a predetermined standard error has been reached for the most recent examinee trait estimate (Boyd, Dodd, & Choi, 2010)....

show abstract

Section: Methodsmentioning

confidence: 99%

mentioning

confidence: 98%

See 1 more Smart Citation

A New Stopping Rule for Computerized Adaptive Testing

Choi

Grady

Dodd

2010

Educational and Psychological Measurement

Self Cite

View full text Add to dashboard Cite

show abstract

“…The administered tests for certain examinees may be undesirably lengthy or short because the required precision cannot be met or few items have improved the precision significantly. Under the unidimensional framework, some studies have been done on using different stopping rules in CAT (Dodd et al 1993), such as the minimum standard error (SE) stopping rule, the minimum information stopping rule (Dodd et al 1989), and the predicted standard error reduction (PSER) stopping rule (Choi et al 2011). Under the multidimensional framework, many previous studies on item selection methods were conducted in fixed-length MCATs (Mulder and van der Linden 2009;Su and Huang 2014;Wang et al 2011a, b;Yao 2011Yao , 2012.…”

Section: Background and Purposementioning

confidence: 99%

Quantitative Psychology Research

Ark

Bolt

Wang

et al. 2015

Springer Proceedings in Mathematics &Amp; Statistics

View full text Add to dashboard Cite

This book series features volumes composed of select contributions from workshops and conferences in all areas of current research in mathematics and statistics, including operation research and optimization. In addition to an overall evaluation of the interest, scientific quality, and timeliness of each proposal at the hands of the publisher, individual contributions are all refereed to the high quality standards of leading journals in the field. Thus, this series provides the research community with well-edited, authoritative reports on developments in the most exciting areas of mathematical and statistical research today.L. Andries van der Ark • Daniel M. Bolt

show abstract

“…Interesting results were, for example, that compared to CAT with dichotomous item scores, the size of the item pool may be substantially smaller to get an accurate estimate of 0 (see e.g., De Ayala, 1993, and. Item pools should be not too small, however, in order to secure, for example, content validity.…”

Section: Polytomous Item Response Modelsmentioning

confidence: 99%

Detection of Person Misfit in Computerized Adaptive Tests with Polytomous Items

Krimpen-Stoop¹,

Meijer

2002

Applied Psychological Measurement

View full text Add to dashboard Cite

Item scores that do not fit an assumed item response theory model may cause the latent trait value to be estimated inaccurately. For computerized adaptive tests (CAT) with dichotomous items, several person-fit statistics for detecting nonfitting item score patterns have been proposed. Both for paper-and-pencil (P&P) test and CATs, detection of person misfit with polytomous items has hardly been explored. In this simulation study, the theoretical and empirical null distributions of a person-fit statistic for polytomous items are compared for P&P tests and CATs. Results show that the empirical distribution of this statistic was close to the standard normal distribution, for both P&P tests and CATs. Also statistics that are especially designed for a CAT are proposed. In these statistics observed and expected item scales are compared using cumulative sum (CUSUM) procedures.

show abstract

Computerized Adaptive Testing Using the Partial Credit Model: Effects Of Item Pool Characteristics and Different Stopping Rules

Cited by 32 publications

References 9 publications

A New Stopping Rule for Computerized Adaptive Testing

A New Stopping Rule for Computerized Adaptive Testing

Quantitative Psychology Research

Detection of Person Misfit in Computerized Adaptive Tests with Polytomous Items

Contact Info

Product

Resources

About