A Nomothetic Span Approach to assessing the Construct Validity of Sustained Attention Measures

Abstract

Sustained attention is frequently assessed using either objective behavioral measures, such as reaction-time variability during a task, or subjective self-report measures, such as rates of task-unrelated thought (TUT) during a task. The current studies examined whether the individual-difference overlap in these measures provides a more construct valid assessment of sustained attention ability than does either alone. Whereas prior research typically considered behavioral measures to provide convergent validity for TUT reports, we argue that they mutually validate each other; each measurement approach has its own sources of error, so their shared variance should best reflect the sustained attention construct. We reanalyzed two latent-variable studies where reaction-time variability and TUTs were measured in multiple tasks (Kane et al., 2016, Journal of Experimental Psychology: General, 145, 1017-1048; Unsworth et al., 2021, Journal of Experimental Psychology: General, 150, 1303-1331). These studies also included several nomological-network constructs that allowed for testing the convergent and discriminant validity of a general sustained attention factor. Confirmatory factor analyses assessing bifactor (preregistered) and hierarchical (non-preregistered) models, suggested that sustained attention can be modeled as the shared variance among objective and subjective measures. This sustained attention factor was related to working memory capacity, attention control, processing speed, task-specific motivation and alertness, and self-reported cognitive failures and positive schizotypy. Multiverse analyses of trial- and subject-level outlier decisions suggested that bifactor models of general sustained attention ability are less robust than hierarchical models. The results provide evidence for the general ability to sustain attention and provide suggestions for improving its measurement.

Naturalistic assessment of reaction time variability in older adults at risk for Alzheimer’s disease

“…We next imputed any RTs for each subject beyond their Median þ 3*Interquartile Range to that value. Thus, we retained as much data for each subject without undue influence of extremely long RTs (see Welhaf & Kane, 2023). This procedure resulted in < 1% of RTs being imputed for each participant.…”

“…The study of cognitive variability over larger timescales (i.e., weeks, months, and years) in healthy aging and AD has revealed that older adults and those at risk for developing AD or already showing AD symptoms are more inconsistent across testing sessions (e.g., Bielak et al, 2019;Cerino et al, 2021;Hultsch et al, 2000;MacDonald & Stawski, 2020;Martin & Hofer, 2004;Stawski et al, 2015). However, moment-to-moment variability (over the course of several seconds, rather than hours) might serve an important role in basic psychological and everyday functional processes (Unsworth & Miller, 2021;Unsworth, 2015;Welhaf & Kane, 2023). For example, the ability to maintain consistent attention and focus over the course of several seconds is important for many everyday functions like remembering a shopping list, driving, following basic instructions, and even focusing during a conversation.…”

Naturalistic Assessment of Reaction Time Variability in Older Adults at Risk for Alzheimer Disease

“…We next imputed any RTs for each subject beyond their Median + 3*Interquartile Range (IQR) to that value. Thus, we retained as much data for each subject without undue influence of extremely long RTs (see Welhaf & Kane, 2023). This procedure resulted in < 1%…”

“…The study of cognitive variability over larger timescales (i.e., weeks, months, and years) in healthy aging and AD has revealed that older adults and those at risk for developing AD or already showing AD symptoms are more inconsistent across testing sessions (e.g., Bielak, Mogle, & Sliwinski, 2019;Cerino et al, 2021;Hultsch et al, 2000;MacDonald & Stawski, 2020;Martin & Hofer, 2004;Stawski, MacDonald, & Sliwinski, 2015). However, moment-to-moment variability (over the course of several seconds, rather than hours) might serve an important role in basic psychological and everyday functional processes (Unsworth, 2015;Unsworth & Miller, 2021;Welhaf & Kane, 2023). For example, the ability to maintain consistent attention and focus over the course of several seconds is important for many everyday functions like remembering a shopping list, driving, following basic instructions, and even focusing during a conversation.…”

Antecedents of mind wandering states in healthy aging and mild cognitive impairment.

“…Behavioral measures of mind wandering, including preerror speeding and reaction time (RT) variability, are often used to supplement self-reports and may actually reflect slightly different aspects of mind wandering (Unsworth et al, 2021; Welhaf & Kane, in press). For example, Unsworth et al (2021) found a latent variable correlation of 0.42 between behavioral and self-reported markers of mind wandering, suggesting an overlap but also a clear separation between constructs.…”