Investigation of Preknowledge Cheating via Joint Hierarchical Modeling Patterns of Response Accuracy and Response Time

Abstract

As computer-based testing becomes more prevalent, the attention paid to response time (RT) in assessment practice and psychometric research correspondingly increases. This study explores the rate of Type I error in detecting preknowledge cheating behaviors, the power of the Kullback-Leibler (KL) divergence measure, and the L person fit statistic under various conditions by modeling patterns of response accuracy (RA) and RT using a joint hierarchical model. Four design factors were manipulated: test length, the difficulty level of compromised items, the ratio of compromised items, and variations in the RTs for these compromised items. The results indicate that the KL measure consistently exhibits higher power and Type I error rates than the person fit statistics [Formula: see text] and [Formula: see text] across all RA and RT patterns and under all conditions. Furthermore, the KL measure demonstrates the greatest power at a medium test length.