Optimal Testing With Easy or Difficult Items in Computerized Adaptive Testing

Abstract

Computerized adaptive tests (CATs) are individualized tests that, from a measurement point of view, are optimal for each individual, possibly under some practical conditions. In the present study, it is shown that maximum information item selection in CATs using an item bank that is calibrated with the one-or the two-parameter logistic model results in each individual answering about 50% of the items correctly. Two item selection procedures giving easier (or more difficult) tests for students are presented and evaluated. Item selection on probability points of items yields good results only with the one-parameter logistic model and not with the two-parameter logistic model. An alternative selection procedure, based on maximum information at a shifted ability level, gives satisfactory results with both models.

Automated Adaptation and Assessment in Serious Games: A Portable Tool for Supporting Learning

“…P t is drawn from a normal distribution N (P = 0.75, SD = 0.1) and restricted such that 0.5 < P t < 1. Such value of P t allows the player to maintain an average success rate of 75% [5,13]. According to these studies, the success rate of P = 0.75 provides a reasonable balance between keeping a player motivated and maintaining measurement accuracies of ratings.…”

Item Selection Methods Based on Multiple Objective Approaches for Classifying Respondents Into Multiple Levels

“…This possibly reduces the examinee's frustration because fewer too easy or too hard items are administered and a larger set of items is selected from the item pool. However, examinees can experience that the items in a CAT are difficult (Eggen & Verschoor, 2006) when compared with a regular test in which an able student answers only relatively easy items. This drawback of CAT as well as CCT was overcome by Eggen and Verschoor (2006) by selecting easy items.…”

“…However, examinees can experience that the items in a CAT are difficult (Eggen & Verschoor, 2006) when compared with a regular test in which an able student answers only relatively easy items. This drawback of CAT as well as CCT was overcome by Eggen and Verschoor (2006) by selecting easy items. CCT also shares the drawback with CAT that examinees cannot change answers to previously administered items.…”

Mathematical Modelling of Forecasting the Results of Knowledge Testing / Žinių Tikrinimo Rezultatų Prognozės Matematinis Modeliavimas

“…individualized tests that are optimal to each individual (Eggen, Verschoor 2006); latent class analysis (LCA) -a statistical method used to identify a set of discrete, mutually exclusive latent classes of individuals, based on their responses to a set of observed categorical variables (Lanza et al 2007); new technologies such as heuristic search and machine learning approaches, including neural networks to automatically identify the most-informative subset of test items, when the item bank is very large (El-Alfy, Abdel-Aal 2008); tests of model misfit to validate the use of a particular model in IRT (Wells, Bolt 2008); evaluation of the standard error of the estimated latent variable score (Hoshino, Shigemasu 2008).…”