Limited information estimation of the diffusion‐based item response theory model for responses and response times

Abstract

Psychological tests are usually analysed with item response models. Recently, some alternative measurement models have been proposed that were derived from cognitive process models developed in experimental psychology. These models consider the responses but also the response times of the test takers. Two such models are the Q-diffusion model and the D-diffusion model. Both models can be calibrated with the diffIRT package of the R statistical environment via marginal maximum likelihood (MML) estimation. In this manuscript, an alternative approach to model calibration is proposed. The approach is based on weighted least squares estimation and parallels the standard estimation approach in structural equation modelling. Estimates are determined by minimizing the discrepancy between the observed and the implied covariance matrix. The estimator is simple to implement, consistent, and asymptotically normally distributed. Least squares estimation also provides a test of model fit by comparing the observed and implied covariance matrix. The estimator and the test of model fit are evaluated in a simulation study. Although parameter recovery is good, the estimator is less efficient than the MML estimator.

Analysing model fit of psychometric process models: An overview, a new test and an application to the diffusion model

“…First, the diffusion model was fitted to the data. As the different tests are based on different estimators, the item parameters were estimated three times: with marginal maximum likelihood estimation (Molenaar et al, 2015) and with unweighted and diagonally weighted least squares estimation (Ranger et al, 2016). For results concerning parameter recovery, see the simulation study of Ranger et al (2016).…”

“…Hence, an analysis of model fit at the subject level requires an estimate of the test taker's latent trait and a statistical approach that accounts for the effect of trait estimation. Last, but not least, the statistical tests of model fit in experimental psychology require specific approaches to model estimation, such as the v 2 approach (Ratcliff & Tuerlinckx, 2002) or the Kolmogorov-Smirnov approach (Voss & Voss, 2008), which are not used in cognitive psychometrics (Molenaar et al, 2015;Ranger, Kuhn, & Szardenings, 2016).…”

“…If the assumed model is valid or provides a good approximation of the true data generation process the sample average should be close to zero. Several tests for the fit of psychometric process modelsamong them those of Molenaar et al (2015), Ranger and Kuhn (2014b) and Ranger et al (2016) can be shown to be variants of the generalized moment test. In addition to these tests we propose a new one, the Hausman test of misspecification (Hausman, 1978).…”

“…Here, least squares estimates (Ranger et al, 2016) are used as this permits the application of the tests developed in structural equation modeling. Using the quantities defined above, the fit can be assessed via the residuals s À rðx ULS Þ, which have an expectation of zero if the model holds.…”

“…It improves upon the former test by considering response times in addition to the responses. The test was implemented for the diffusion model by Ranger et al (2016). Ranger and Kuhn (2014b) proposed a general test of model fit for response and response time models.…”