Initial Psychometric Properties of 7 NeuroUX Remote Ecological Momentary Cognitive Tests Among People With Bipolar Disorder: Validation Study

Abstract

Background As smartphone technology has become nearly ubiquitous, there is a growing body of literature suggesting that ecological momentary cognitive testing (EMCT) offers advantages over traditional pen-and-paper psychological assessment. We introduce a newly developed platform for the self-administration of cognitive tests in ecologically valid ways. Objective The aim of this study is to develop a Health Insurance Portability and Accountability Act–compliant EMCT smartphone-based platform for the frequent and repeated testing of cognitive abilities in everyday life. This study examines the psychometric properties of 7 mobile cognitive tests covering domains of processing speed, visual working memory, recognition memory, and response inhibition within our platform among persons with and without bipolar disorder (BD). Ultimately, if shown to have adequate psychometric properties, EMCTs may be useful in research on BD and other neurological and psychiatric illnesses. Methods A total of 45 persons with BD and 21 demographically comparable healthy volunteer participants (aged 18-65 years) completed smartphone-based EMCTs 3 times daily for 14 days. Each EMCT session lasted approximately 1.5 minutes. Only 2 to 3 tests were administered in any given session, no test was administered more than once per day, and alternate test versions were administered in each session. Results The mean adherence to the EMCT protocol was 69.7% (SD 20.5%), resulting in 3965 valid and complete tests across the full sample. Participants were significantly more likely to miss tests on later versus earlier study days. Adherence did not differ by diagnostic status, suggesting that BD does not interfere with EMCT participation. In most tests, age and education were related to EMCT performance in expected directions. The average performances on most EMCTs were moderately to strongly correlated with the National Institutes of Health Toolbox Cognition Battery. Practice effects were observed in 5 tests, with significant differences in practice effects by BD status in 3 tests. Conclusions Although additional reliability and validity data are needed, this study provides initial psychometric support for EMCTs in the assessment of cognitive performance in real-world contexts in BD.

Feasibility and validity of ecological momentary cognitive testing among older adults with mild cognitive impairment

“…We found mixed findings of a fatigue effect at the level of the individual tests, such that there was no evidence of a fatigue effect for the VLMT or Color Trick tests, but participants were more likely to miss Memory Matrix tests over the course of the 30-day protocol (with no difference by NC vs MCI). In another study using the VLMT and Memory Matrix test (14-day protocol in participants with bipolar disorder and control participants) we found an overall fatigue effect for the EMCT protocol, such that participants were more likely to miss a test as study day increased (no differences by diagnostic status), but we did not examine fatigue effects at the level of the individual test (30). Of note, the prior study had a more intensive protocol than the current study, with participants pinged to complete 2-3 mobile cognitive tests three times daily for 14-days.…”

Assessing cancer-related cognitive function in the context of everyday life using ecological mobile cognitive testing: A protocol for a prospective quantitative study

“…NeuroUX EMCT has demonstrated reliability and validity in middle-aged and older adults, with and without cognitive impairment. 39 , 42 , 43 …”

“…Use of EMCT in research studies is on the rise, and feasibility, validity, and reliability of EMCT has been established in several cognitively vulnerable populations, including people with serious mental illness, mood disorders, and mild cognitive impairment. 39 – 43 However, EMCT studies in the field of CRCI are only recently gaining traction.…”

Self-regulatory control processes in youths: A temporal network analysis approach

“…Ambulatory cognitive performance (Jones et al, 2018) is also embedded in ambulatory assessment even if it is less common in previous studies. Such assessment provides valid and reliable measures of cognitive control (Moore et al, 2022).…”