Response moderation models for conditional dependence between response time and response accuracy

Abstract

It is becoming more feasible and common to register response times in the application of psychometric tests. Researchers thus have the opportunity to jointly model response accuracy and response time, which provides users with more relevant information. The most common choice is to use the hierarchical model (van der Linden, 2007, Psychometrika, 72, 287), which assumes conditional independence between response time and accuracy, given a person's speed and ability. However, this assumption may be violated in practice if, for example, persons vary their speed or differ in their response strategies, leading to conditional dependence between response time and accuracy and confounding measurement. We propose six nested hierarchical models for response time and accuracy that allow for conditional dependence, and discuss their relationship to existing models. Unlike existing approaches, the proposed hierarchical models allow for various forms of conditional dependence in the model and allow the effect of continuous residual response time on response accuracy to be item-specific, person-specific, or both. Estimation procedures for the models are proposed, as well as two information criteria that can be used for model selection. Parameter recovery and usefulness of the information criteria are investigated using simulation, indicating that the procedure works well and is likely to select the appropriate model. Two empirical applications are discussed to illustrate the different types of conditional dependence that may occur in practice and how these can be captured using the proposed hierarchical models.

A Speed-Accuracy Tradeoff Hierarchical Model Based on Cognitive Experiment

“…Therefore, the effect of residual response time is more likely to be negative for medium-difficulty items. The result is slightly different from that of Bolsinova et al (2017; Figure 3). It may be related to the difficulty of the test, because the test is relatively simple.…”

“…In order to compare the SAT response model with other models, response moderation model was Eq. 6 (RMM, Bolsinova et al, 2017). Figure 1 showed the relationship between residual time and accuracy of SATM and RMM.…”

“…The person parameters θ and τ were sampled from a bivariate normal distribution with σ θτ = 0.5. The chosen parameters, test length and sample size are the most commonly used settings (Wang et al, 2013;Meng et al, 2015;Bolsinova et al, 2017).…”

“…However, they cannot directly reflect the dynamic relationship between response time and accuracy. In contrast with covariance, a response moderation model took the residual response time as the independent variable of the response model (Bolsinova et al, 2016(Bolsinova et al, , 2017. Nevertheless, the response moderation model enlarged the time effect and ignored the influence of ability on accuracy.…”

Improving precision of ability estimation: Getting more from response times

“…However, the correlation that is observed between speed and ability in both low-and high-stakes tests may vary and is not guaranteed to be non-zero; see, for, example the analyses of empirical data sets of Bolsinova, de Boeck, and Tijmstra (2017), Bolsinova, Tijmstra, and Molenaar (2016), and van der Linden, Scrams, and Schnipke (1999), who found negligible correlations between ability and speed. The speed parameter that is estimated in the HM represents the effective speed of a person as displayed on that particular test.…”