The stochastically curtailed generalized likelihood ratio: A new termination criterion for variable-length computerized classification tests

Abstract

Computerized classification tests (CCTs) are used to classify examinees into categories in the context of professional certification testing. The term "variable-length" refers to CCTs that terminate (i.e., cease administering items to the examinee) when a classification can be made with a prespecified level of certainty. The sequential probability ratio test (SPRT) is a common criterion for terminating variable-length CCTs, but recent research has proposed more efficient methods. Specifically, the stochastically curtailed SPRT (SCSPRT) and the generalized likelihood ratio criterion (GLR) have been shown to classify examinees with accuracy similar to the SPRT while using fewer items. This article shows that the GLR criterion itself may be stochastically curtailed, resulting in a new termination criterion, the stochastically curtailed GLR (SCGLR). All four criteria-the SPRT, SCSPRT, GLR, and the new SCGLR-were compared using a simulation study. In this study, we examined the criteria in testing conditions that varied several CCT design features, including item bank characteristics, pass/fail threshold, and examinee ability distribution. In each condition, the termination criteria were evaluated according to their accuracy (proportion of examinees classified correctly), efficiency (test length), and loss (a single statistic combing both accuracy and efficiency). The simulation results showed that the SCGLR can yield increased efficiency without sacrificing accuracy, relative to the SPRT, SCSPRT, and GLR in a wide variety of CCT designs.

Potential of Allogeneic Adipose-Derived Stem Cell–Hydrogel Complex for Treating Diabetic Foot Ulcers

“…In this study, Z 0.975 was 1.96 and Z 0.80 was 0.84. Therefore, the number of subjects in each group could be calculated as follows (15): Considering a dropout rate of broadly 10%, the number of subjects in each group was 26.97 (= 24.28/(1 2 0.10)), and a sample size of 54 (= 27 3 2) randomly assigned subjects was required. Based on our experience from previous clinical trials for treating diabetic foot ulcers, ;10% of subjects experienced unexpected events related to diabetesrelated complications and high-risk comorbidities.…”

Stochastic Curtailment in Adaptive Mastery Testing

“…Previous research (Finkelman, 2008(Finkelman, , 2010 had demonstrated the success of both methods when the TSPRT stopping rule is used. More recently, stochastic curtailment was also applied in conjunction with the GLR stopping rule and was shown to yield increased efficiency while maintaining similar accuracy (Huebner & Fina, 2014). However, as explained in the Introduction, no previous research had applied curtailment or stochastic curtailment to the CI stopping rule, which is preferable to the TSPRT and the GLR in some testing contexts.…”

On Computing the Key Probability in the Stochastically Curtailed Sequential Probability Ratio Test

“…The authors note that although Kullback-Leibler information has also been studied as an alternative to FI, the two item selection criteria perform very similarly in terms of efficiency and accuracy in practical testing situations (Lin, 2011). Moreover, recent studies on SCSPRT methodology (Finkelman, 2008(Finkelman, , 2010Huebner & Fina, 2015) use the FI index. Also, to mimic a practical testing situation, the Sympson-Hetter method of item exposure control (Sympson & Hetter, 1985) was used with a desired maximum exposure rate of 0.20.…”