A combined experimental–correlational approach to the construct validity of performance-based and self-report-based measures of sustained attention

Abstract

The ability to sustain attention is often measured with either objective performance indicators, like within-person RT variability, or subjective self-reports, like mind wandering propensity. A more construct valid approach, however, may be to assess the covariation in these performance and selfreport measures, given that each of these is influenced by different sources of measurement error. If the correlation between performance-variability and self-report measures reflects the sustained attention construct, then task manipulations aimed at reducing the sustained attention demands of tasks should reduce the correlation between them (in addition to reducing mean levels of variability and mind wandering). The current study investigated this claim with a combined experimentalcorrelation approach. In two experiments (Ns ~ 1500 each), participants completed tasks that either maximized or minimized the demand for sustained attention. Our demand manipulations successfully reduced the mean levels of sustained attention failures in both the objective and subjective measures, in both experiments. In neither experiment, however, did the covariation between these measures change as a function of the sustained attention demands of the tasks. We can therefore claim only minimal support for the construct validity of our measurement approach to sustained attention.

Effects of emotional valence of mind wandering on sustained attention performance.

“…There are a few notable results of this analysis. First, overall, TUTs were modestly related to RTsd ( r = .17 [0.14, 0.21]) consistent with previous research (e.g., Unsworth et al, 2021; Welhaf & Kane, 2023a, 2023b). People who reported being off-task more frequently in the SART also showed more inconsistent responding.…”

“…Cleaning of SART performance data was handled using the same procedure for each study (note this might differ from data cleaning procedures either reported or unreported in previous papers). First, to ensure participants engaged with the task sufficiently and understood instructions, we removed anyone who had <70% accuracy on the frequent “go” trials (Welhaf & Kane, 2023a, 2023b).…”