Systematic and random sources of variability in perceptual decision-making: Comment on Ratcliff, Voskuilen, and McKoon (2018).

Abstract

A key assumption of models of human cognition is that there is variability in information processing. Evidence accumulation models (EAMs) commonly assume two broad variabilities in information processing: within-trial variability, which is thought to reflect moment-to-moment fluctuations in perceptual processes, and between-trial variability, which is thought to reflect variability in slower-changing processes like attention, or systematic variability between the stimuli on different trials. Recently, Ratcliff, Voskuilen, and McKoon (2018) claimed to "provide direct evidence that external noise is, in fact, required to explain the data from five simple two-choice decision tasks" (p. 33), suggesting that at least some portion of the between-trial variability in information processing is due to "noise". However, we argue that Ratcliff et al. (2018) failed to distinguish between two different potential sources of between-trial variability: random (i.e., "external noise") and systematic (e.g., item effects). Contrary to the claims of Ratcliff et al. (2018), we show that "external noise" is not required to explain their findings, as the same trends of data can be produced when only item effects are present. Furthermore, we contend that the concept of "noise" within cognitive models merely serves as a convenience parameter for sources of variability that we know exist, but are unable to account for. Therefore, we question the usefulness of experiments aimed at testing the general existence of "random" variability, and instead suggest that future research should attempt to replace the random variability terms within cognitive models with actual explanations of the process.

The gated cascade diffusion model: An integrated theory of decision-making, motor preparation, and motor execution

“…Although between-trial variability in processing components is plausible, we believe that a large contribution of between-trial variability to the fit quality of a model is problematic, as there is generally no explanation of why this variability occurs or why it has the parametric form researchers assume to represent it (see the modeling of Experiment 2 for an exception regarding between-trial variability in drift rate that arises from theoretical properties of the approximate number system). In this view, between-trial variability essentially corresponds to adding a random component to the model without any strong theoretical motivation for it rather than to improve the fit quality (Evans, Tillman, et al, 2020). Consequently, we consider our findings regarding between-trial variability as additional evidence for DSDTDM.…”

Inclusion Bayes Factors for Mixed Hierarchical Diffusion Decision Models

“…The third elaboration, which completes the specification of the full DDM that we focus on here, is uniform across-trial variability in non-decision time with mean t 0 and range s t (Ratcliff & Tuerlinckx, 2002). Although these three elaborations make the DDM a realistic model of choice RT, the across-trial variability parameters can be hard to estimate (Boehm, Annis, et al, 2018;Evans et al, 2020;Lerche et al, 2017;van Ravenzwaaij & Oberauer, 2009). Distribution functions for the full DDM can be obtained analytically for across-trial drift variability and through numerical integration over the non-decision time and start-point distributions.…”

A model-based approach to disentangling facilitation and interference effects in conflict tasks.

“…It should also be noted that the diffusion framework often includes three additional parameters for the between-trial variability in drift rate, starting point, and nondecision time, respectively (Ratcliff, 1978; Ratcliff & Rouder, 1998; Ratcliff & Tuerlinckx, 2002), and that these parameters can be integrated into the DMC framework (Evans & Servant, 2020). However, these between-trial variability parameters can compromise the measurement properties of the model (Boehm et al, 2018; Evans, Tillman, & Wagenmakers, 2020; Lerche et al, 2017; Lerche & Voss, 2016; van Ravenzwaaij & Oberauer, 2009), which would prevent us from robustly disentangling facilitation and interference effects.…”