A simplified version of the maximum information per time unit method in computerized adaptive testing

Cheng, Ying; Diao, Qian; Behrens, John T.

doi:10.3758/s13428-016-0712-6

Cited by 19 publications

(32 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The simulation study of the present research included conditions in which the item difficulty parameter was positively correlated with the time-intensity parameter. As pointed out by Cheng et al (2017), the difficulty parameter is also frequently positively correlated with the discrimination parameter. Hence, the , and time-intensity parameters might all be positively associated; and in fact, Cheng et al did find a positive correlation between such parameters empirically.…”

Section: Discussionmentioning

confidence: 79%

“…When a parametric model does not fit the empirical response-time distribution well, a semi-parametric model that has weaker assumptions could be used instead (e.g., Wang, Fan, Chang, & Douglas, 2013). Also, as mentioned previously, Cheng et al (2017) provided a version of information per time unit that does not require the fitting of a response-time model, and the time-based item selection procedure discussed in the present study can be used within their framework. As the desire to obtain timeefficient information might be present in educational, psychological, and medical assessment, and respondents might tend to answer some items more quickly than others in any of these contexts (differences in cognitive processes and motivations notwithstanding), Fisher information per time unit might prove to be a valuable tool to gain information quickly in CAT applications including diverse uses of AMC.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Time-Efficient Adaptive Measurement of Change

Finkelman¹,

Wang²

2019

JCAT

View full text Add to dashboard Cite

The adaptive measurement of change (AMC) refers to the use of computerized adaptive testing (CAT) at multiple occasions to efficiently assess a respondent's improvement, decline, or sameness from occasion to occasion. Whereas previous AMC research focused on administering the most informative item to a respondent at each stage of testing, the current research proposes the use of Fisher information per time unit as an item selection procedure for AMC. The latter procedure incorporates not only the amount of information provided by a given item but also the expected amount of time required to complete it. In a simulation study, the use of Fisher information per time unit item selection resulted in a lower false positive rate in the majority of conditions studied, and a higher true positive rate in all conditions studied, compared to item selection via Fisher information without accounting for the expected time taken. Future directions of research are suggested.

show abstract

Section: Discussionmentioning

confidence: 79%

Section: Discussionmentioning

confidence: 99%

Time-Efficient Adaptive Measurement of Change

Finkelman¹,

Wang²

2019

JCAT

View full text Add to dashboard Cite

show abstract

“…Coinciding with previous results, the highest correlations were seen in the FL condition as well as in the SE [.30 Item exposure, pool utilization, and content coverage Table 5 presents descriptives of item exposure, pool utilization, and content area coverage. The majority of stopping rule conditions produced mean and maximum item exposure rates that were less than the commonly used target maximum exposure rates of 0.2 (e.g., Cheng, Diao, & Behrens, 2017;Wang, Chang, & Douglas, 2012) and 0.3 (e.g., Leroux & Dodd, 2014;Moyer, Galindo, & Dodd, 2012). The exposure rates closely aligned with the number of items administered (Table 3).…”

Section: Latent Trait Recoverymentioning

confidence: 74%

Comparing computer adaptive testing stopping rules under the generalized partial-credit model

et al. 2018

View full text Add to dashboard Cite

An important consideration of any computer adaptive testing (CAT) program is the criterion used for ending item administration-the stopping rule, which ensures that all examinees are assessed to the same standard. Although various stopping rules exist, none of them have been compared under the generalized partial-credit model (Muraki in Applied Psychological Measurement, 16, 159-176, 1992). In this simulation study we compared the performance of three variable-length stopping rules-standard error (SE), minimum information (MI), and change in theta (CT)-both in isolation and in combination with requirements of minimum and maximum numbers of items, as well as a fixed-length stopping rule. Each stopping rule was examined under two termination criteria-one a more lenient requirement (SE = 0.35, MI = 0.56, CT = 0.05), and one more stringent (SE = 0.30, MI = 0.42, CT = 0.02). The simulation design also included content-balancing and exposure controls, aspects of CAT that have been excluded in previous research comparing variable-length stopping rules. The minimum-information stopping rule produced biased theta estimates and varied greatly in measurement quality across the theta distribution. The absolute-change-in-theta stopping rule had strong performance when paired with a lower criterion and a minimum test length. The standard error stopping rule consistently provided the best balance of measurement precision and operational efficiency and was based on the fewest number of administered items necessary to obtain accurate and precise theta estimates, particularly when it was paired with a maximum-number-of-items stopping rule.

show abstract

“…An immediate implication for the follow-up adaptive design of the AM-PAC is that RT does not need to be included in interim θ estimation (i.e., selecting items during assessment delivery), but it could be used to improve the final θ estimates. Moreover, to further improve the time efficiency of adaptive testing, the maximum information per time unit (Fan et al, 2012) or its simplified version (Cheng et al, 2017) could be applied. In this case, the interviewer effect could be ignored when estimating an individual patient's speed, as long as item time parameters are provided.…”

Section: Discussionmentioning

confidence: 99%

Modeling Response Time and Responses in Multidimensional Health Measurement

Wang

Weiss

2019

Front. Psychol.

View full text Add to dashboard Cite

This study explored calibrating a large item bank for use in multidimensional health measurement with computerized adaptive testing, using both item responses and response time (RT) information. The Activity Measure for Post-Acute Care is a patient-reported outcomes measure comprised of three correlated scales (Applied Cognition, Daily Activities, and Mobility). All items from each scale are Likert type, so that a respondent chooses a response from an ordered set of four response options. The most appropriate item response theory model for analyzing and scoring these items is the multidimensional graded response model (MGRM). During the field testing of the items, an interviewer read each item to a patient and recorded, on a tablet computer, the patient's responses and the software recorded RTs. Due to the large item bank with over 300 items, data collection was conducted in four batches with a common set of anchor items to link the scale. van der Linden's (2007) hierarchical modeling framework was adopted. Several models, with or without interviewer as a covariate and with or without interaction between interviewer and items, were compared for each batch of data. It was found that the model with the interaction between interviewer and item, when the interaction effect was constrained to be proportional, fit the data best. Therefore, the final hierarchical model with a lognormal model for RT and the MGRM for response data was fitted to all batches of data via a concurrent calibration. Evaluation of parameter estimates revealed that (1) adding response time information did not affect the item parameter estimates and their standard errors significantly; (2) adding response time information helped reduce the standard error of patients' multidimensional latent trait estimates, but adding interviewer as a covariate did not result in further improvement. Implications of the findings for follow up adaptive test delivery design are discussed.

show abstract

A simplified version of the maximum information per time unit method in computerized adaptive testing

Cited by 19 publications

References 27 publications

Time-Efficient Adaptive Measurement of Change

Time-Efficient Adaptive Measurement of Change

Comparing computer adaptive testing stopping rules under the generalized partial-credit model

Modeling Response Time and Responses in Multidimensional Health Measurement

Contact Info

Product

Resources

About