Reliability and Validity of Noncognitive Ecological Momentary Assessment Survey Response Times as an Indicator of Cognitive Processing Speed in People’s Natural Environment: Intensive Longitudinal Study

Abstract

Background Various populations with chronic conditions are at risk for decreased cognitive performance, making assessment of their cognition important. Formal mobile cognitive assessments measure cognitive performance with greater ecological validity than traditional laboratory-based testing but add to participant task demands. Given that responding to a survey is considered a cognitively demanding task itself, information that is passively collected as a by-product of ecological momentary assessment (EMA) may be a means through which people’s cognitive performance in their natural environment can be estimated when formal ambulatory cognitive assessment is not feasible. We specifically examined whether the item response times (RTs) to EMA questions (eg, mood) can serve as approximations of cognitive processing speed. Objective This study aims to investigate whether the RTs from noncognitive EMA surveys can serve as approximate indicators of between-person (BP) differences and momentary within-person (WP) variability in cognitive processing speed. Methods Data from a 2-week EMA study investigating the relationships among glucose, emotion, and functioning in adults with type 1 diabetes were analyzed. Validated mobile cognitive tests assessing processing speed (Symbol Search task) and sustained attention (Go-No Go task) were administered together with noncognitive EMA surveys 5 to 6 times per day via smartphones. Multilevel modeling was used to examine the reliability of EMA RTs, their convergent validity with the Symbol Search task, and their divergent validity with the Go-No Go task. Other tests of the validity of EMA RTs included the examination of their associations with age, depression, fatigue, and the time of day. Results Overall, in BP analyses, evidence was found supporting the reliability and convergent validity of EMA question RTs from even a single repeatedly administered EMA item as a measure of average processing speed. BP correlations between the Symbol Search task and EMA RTs ranged from 0.43 to 0.58 (P<.001). EMA RTs had significant BP associations with age (P<.001), as expected, but not with depression (P=.20) or average fatigue (P=.18). In WP analyses, the RTs to 16 slider items and all 22 EMA items (including the 16 slider items) had acceptable (>0.70) WP reliability. After correcting for unreliability in multilevel models, EMA RTs from most combinations of items showed moderate WP correlations with the Symbol Search task (ranged from 0.29 to 0.58; P<.001) and demonstrated theoretically expected relationships with momentary fatigue and the time of day. The associations between EMA RTs and the Symbol Search task were greater than those between EMA RTs and the Go-No Go task at both the BP and WP levels, providing evidence of divergent validity. Conclusions Assessing the RTs to EMA items (eg, mood) may be a method of approximating people’s average levels of and momentary fluctuations in processing speed without adding tasks beyond the survey questions.

Reliability and Validity of Ecological Momentary Assessment Response Time–Based Measures of Emotional Clarity: Secondary Data Analysis

“…In addition, covariances were specified between baseline speed and all other level-2 variables. Prior research indicated that RTs to tasks were affected by the time of day [ 19 , 56 ]. Therefore, at level 1, all the RT-based metrics were adjusted for (ie, regressed on) the time of day, coded as a categorical variable, where a participant’s first survey per day was categorized as taking place in the morning, their final scheduled survey of the day as taking place in the evening, and all their surveys in between as taking place midday.…”

“…Finally, the possibility that processing speed had a confounding effect on correlations between study-long aggregates of emotion item RTs and other variables was not investigated. In prior research, RTs to emotion EMA items have been found to have moderate correlations with processing speed measures [ 19 ], suggesting that individual differences in emotion item RTs may at least in part be attributable to processing speed.…”

“…We examined whether adjusting the RT-based emotional clarity indicators by individual differences in RT and participant processing speed would affect the results of the convergent or divergent validity tests. Both RTs to survey items and drift rate were expected to not be purely indicators of emotional clarity but likely also be impacted by processing speed [ 19 , 20 , 37 ] and differences in the baseline speed of responding [ 12 ]. Therefore, we examined the robustness of the results of validity testing when statistically adjusting for both these variables.…”

Using a Novel Digital Go/No-Go to Dissociate Intra-subject Temporal Fluctuations in Reaction Time and Accuracy

“…To illustrate the importance of taking a naturalistic in-situ approach, previous studies deploying these neurocognitive tasks have primarily administered them during in-person visits, which is limited by the frequency of visits and likely restricts the hours in which these tasks can be completed and increases the risk of performance anxiety. With the increasing ubiquity of smartphones, researchers have been able to adapt neurocognitive tasks, such as the Psychomotor Vigilance Test (PVT) [49,50], the GNG [51][52][53], and The Flanker Task [4,54], to smartphones to measure individual changes in reaction times, inhibitory control, and alertness in-the-wild. To this end, a few studies in human-computer interaction (HCI) have used these digital adaptations to measure time-of-day fluctuations in cognitive processes, specifically alertness.…”