Bayesian hierarchical response time modelling—A tutorial

Abstract

Response time modelling is developing rapidly in the field of psychometrics, and its use is growing in psychology. In most applications, component models for response times are modelled jointly with component models for responses, thereby stabilizing estimation of item response theory model parameters and enabling research on a variety of novel substantive research questions. Bayesian estimation techniques facilitate estimation of response time models. Implementations of these models in standard statistical software, however, are still sparse. In this accessible tutorial, we discuss one of the most common response time models—the lognormal response time model—embedded in the hierarchical framework by van der Linden (2007). We provide detailed guidance on how to specify and estimate this model in a Bayesian hierarchical context. One of the strengths of the presented model is its flexibility, which makes it possible to adapt and extend the model according to researchers' needs and hypotheses on response behaviour. We illustrate this based on three recent model extensions: (a) application to non‐cognitive data incorporating the distance‐difficulty hypothesis, (b) modelling conditional dependencies between response times and responses, and (c) identifying differences in response behaviour via mixture modelling. This tutorial aims to provide a better understanding of the use and utility of response time models, showcases how these models can easily be adapted and extended, and contributes to a growing need for these models to answer novel substantive research questions in both non‐cognitive and cognitive contexts.

Controlling the Speededness of Assembled Test Forms: A Generalization to the Three‐Parameter Lognormal Response Time Model

“…For an extensive tutorial on Baysian hierarchical response time modelling using Stan, see König et al. (2023). Alternatively, the hierarchical framework can also be translated to a frequentist factor analysis framework, as illustrated by Molenaar et al.…”

“…Hierarchical implementation, which can aid measurement on the response and response time side (van der Linden, 2007;van der Linden et al, 2010), can be implemented using, for example, the Rpackage LNIRT (Fox et al, 2021) or general purpose Bayesian estimation software such as Stan (Carpenter et al, 2017). For an extensive tutorial on Baysian hierarchical response time modelling using Stan, see König et al (2023). Alternatively, the hierarchical framework can also be translated to a frequentist factor analysis framework, as illustrated by Molenaar et al (2015).…”

“…For model estimation, we use the default options set by rstan, with 4 chains, 2,000 iterations each and a burn-in of 1,000 iterations. Model specifications adopt the recommendations by König et al (2023) with hierarchical prior specifications and informative or weakly informative priors aiding estimation stability. For instance, for covariance matrices, a separation strategy is utilized, with Cauchy distributions as a prior for standard deviations and an LKJ prior distribution for the Cholesky factor of the correlation matrix.…”

The “neglecting the vectorization" error in Stan: erroneous coding practices for computing marginal likelihood and Bayes factors in models with vectorized truncated distributions

“…It has been widely utilized in the social, biological, and physical sciences, engineering, and business (Stan Development Team 2023). Moreover, a number of articles on statistical modeling using Stan have been published in academic journals (Ames and Au 2018;Annis et al 2017;Jiang and Carter 2019;Koenig et al 2023;Luo and Jiao 2018).…”