Modeling Conditional Dependence of Response Accuracy and Response Time with the Diffusion Item Response Theory Model

Abstract

In this paper, we propose a model-based method to study conditional dependence between response accuracy and response time (RT) with the diffusion IRT model (Tuerlinckx & De Boeck, 2005;van der Maas, Molenaar, Maris, Kievit, & Borsboom, 2011). We extend the earlier diffusion IRT model by introducing variability across persons and items in cognitive capacity (drift rate in the evidence accumulation process) and variability in the starting point of the decision processes. We show that the extended model can explain the behavioral patterns of conditional dependency found in the previous studies in psychometrics. Variability in cognitive capacity can predict positive and negative conditional dependency and their interaction with the item difficulty. Variability in starting point can account for the early changes in the response accuracy as a function of RT given the person and item effects. By the combination of the two variability components, the extended model can produce the curvilinear conditional accuracy functions that have been observed in psychometric data. We also provide a simulation study to validate the parameter recovery of the proposed model and present two empirical applications to show how to implement the model to study conditional dependency underlying data response accuracy and RTs.

Speed–Accuracy Trade-Off? Not So Fast: Marginal Changes in Speed Have Inconsistent Relationships With Accuracy in Real-World Settings

“…Our results also suggest, dovetailing with others, that accuracy is integral for understanding individual differences (i.e., RT alone may be insufficient; Draheim et al, 2019; Draheim et al, 2020). Our approach—emphasizing multiple data sets and nonparametric models—could also be incorporated into further tests of newly developed models (e.g., Kang et al, 2021). Future work could also examine the degree to which our approach could be used as a test of whether models that make restrictions on the SAT are appropriate; for example, identification of a fairly flat curve using the approach of Figure 3 could be a positive sign that the hierarchical model (van der Linden, 2007) could be used.…”

Cognitive Diagnosis Modeling Incorporating Response Times and Fixation Counts: Providing Comprehensive Feedback and Accurate Diagnosis

“…In practice, users need to be aware of whether their data meet these assumptions. In recent years, several studies have been conducted to explore the issue of testing or modeling the local independence assumptions in the joint model but mainly focused on the joint model for RA and RT data (e.g., Bolsinova & Maris, 2016; Bolsinova & Tijmstra, 2016, 2018; Kang et al, 2022; Meng et al, 2015; Zhan, Liao, & Bian, 2018). In future studies, researchers may explore whether these methods can be introduced or extended to the models proposed in this study.…”

A Recent Development of a Network Approach to Assessment Data: Latent Space Item Response Modeling for Intelligence Studies

“…Some prior research has outlined the theoretical mechanisms behind the emergence of CD between responses and RTs, such as speed-accuracy trade-offs and fast/slow guessing (e.g., Bolsinova et al 2017b;De Boeck et al 2017). Furthermore, there have been studies that combine mathematical modeling and psychological measurement models to examine some of these theoretical mechanisms based on formal models and data analysis (e.g., Kang et al 2022aKang et al , 2022b. The recently proposed Latent Space Diffusion Item Response Theory Model (Kang et al 2023) employs a similar approach to LSIRM; It analyzes variations in responses and RTs from psychological/educational tests based on cognitive processes and cognitive components involved in decision-making/problem-solving processes, and simultaneously, attempts to capture and visualize CD between item responses, between RTs, and between responses and RTs through latent space and distance effects.…”

“…In this case, the model has the product term a i • θ p , which captures person-item interaction effects in a systematic way as a combination of the main person and item effects (similar to a moderation/interaction effect in regression models). However, actual interactions may not be fully expressed by these main effect terms, and previous studies on CD acknowledged that unexplained interactions and CD may persist even with the product term implemented in a measurement model (e.g., Kang et al 2022bKang et al , 2023.…”