A spurious correlation between difference scores in evidence-accumulation model parameters

Abstract

Evidence-accumulation models are a useful tool for investigating the cognitive processes that give rise to behavioural data patterns in reaction times (RTs) and error rates. In their simplest form, evidence-accumulation models include three parameters: The average rate of evidence accumulation over time (drift rate) and the amount of evidence that needs to be accumulated before a response becomes selected (boundary) both characterise the response-selection process; a third parameter summarises all processes before and after the response-selection process (non-decision time). Researchers often compute experimental effects as simple difference scores between two within-subject conditions and such difference scores can also be computed on model parameters. In the present paper, we report spurious correlations between such model parameter difference scores, both in empirical data and in computer simulations. The most pronounced spurious effect is a negative correlation between boundary difference and non-decision difference, which amounts to r = – .70 or larger. In the simulations, we only observed this spurious negative correlation when either (a) there was no true difference in model parameters between simulated experimental conditions, or (b) only drift rate was manipulated between simulated experimental conditions; when a true difference existed in boundary separation, non-decision time, or all three main parameters, the correlation disappeared. We suggest that care should be taken when using evidence-accumulation model difference scores for correlational approaches because the parameter difference scores can correlate in the absence of any true inter-individual differences at the population level.

Non-decision time: the Higg’s boson of decision

“…Even if researchers are not interested in NDT itself, not knowing when it should be constrained or not has consequences on other parameter estimates. For instance, it has recently been shown that estimates of between-condition differences in NDT and boundary separation (meant to capture caution) are negatively correlated (Grange & Schuch, 2021). This occurred in simulations when there were no real differences in either parameter.…”

Task Switching: On the Relation of Cognitive Flexibility with Cognitive Capacity

“…The situation looks different, though, when difference scores are computed between two conditions, as has been traditionally done when estimating effects of executive control (e.g., Miyake et al 2000;Stahl et al 2014). In fact, there is evidence that difference scores computed from diffusion model parameters are particularly spurious (Grange and Schuch 2022) which impedes testing their relations with correlates of interest. The lack of reliability stems from at least two sources.…”

“…Such reliability issues may be less of a problem in experimental research, where sufficient data is available from all participants to derive reliable mean estimates for the experimental conditions. However, it can turn out to be a challenge when individual differences are concerned (Grange and Schuch 2022;Schmiedek et al 2007). One possible solution is to treat parameter estimates as error-prone indicators of latent parameter factors that can be modeled by means of structural equation modeling.…”