Unsupervised high-frequency smartphone-based cognitive assessments are reliable, valid, and feasible in older adults at risk for Alzheimer’s disease

Abstract

Objective: Smartphones have the potential for capturing subtle changes in cognition that characterize preclinical Alzheimer’s disease (AD) in older adults. The Ambulatory Research in Cognition (ARC) smartphone application is based on principles from ecological momentary assessment (EMA) and administers brief tests of associative memory, processing speed, and working memory up to 4 times per day over 7 consecutive days. ARC was designed to be administered unsupervised using participants’ personal devices in their everyday environments. Methods: We evaluated the reliability and validity of ARC in a sample of 268 cognitively normal older adults (ages 65–97 years) and 22 individuals with very mild dementia (ages 61–88 years). Participants completed at least one 7-day cycle of ARC testing and conventional cognitive assessments; most also completed cerebrospinal fluid, amyloid and tau positron emission tomography, and structural magnetic resonance imaging studies. Results: First, ARC tasks were reliable as between-person reliability across the 7-day cycle and test-retest reliabilities at 6-month and 1-year follow-ups all exceeded 0.85. Second, ARC demonstrated construct validity as evidenced by correlations with conventional cognitive measures (r = 0.53 between composite scores). Third, ARC measures correlated with AD biomarker burden at baseline to a similar degree as conventional cognitive measures. Finally, the intensive 7-day cycle indicated that ARC was feasible (86.50% approached chose to enroll), well tolerated (80.42% adherence, 4.83% dropout), and was rated favorably by older adult participants. Conclusions: Overall, the results suggest that ARC is reliable and valid and represents a feasible tool for assessing cognitive changes associated with the earliest stages of AD.

Unsupervised mobile app-based cognitive testing in a population-based study of older adults born 1944

“…However, our study did not use a composite measure but individual memory scores. As expected, they were most strongly associated with conventional memory measures (r = 0.27-0.40), which align nicely with relationships of pure memory measures in the abovementioned studies (r = 0.30-0.47 and −0.22-0.32) (15,19). Thus, our results support the notion that the mobile app-based memory measures assess relevant cognitive functions as confirmed by conventional cognitive outcomes but also showed associations with speed and executive function.…”

“…Another recent study ( 34 ) used a composite of similar but not identical tasks as used in this study and demonstrated a strong correlation to PACC ( r = 0.51) in cognitively normal individuals without cognitive complaints. A study using the Ambulatory Research in Cognition app ( 15 ) in primarily cognitively normal participants (again using a composite of cognitive measures) reported a strong correlation with a global composite similar to the PACC ( r = −0.53). Our study found that our mobile app-based tasks were significantly associated with the PACC5 ( r = 0.3–0.5), with similar effect sizes reported in previous work ( 15 , 19 , 34 ).…”

“…A study using the Ambulatory Research in Cognition app ( 15 ) in primarily cognitively normal participants (again using a composite of cognitive measures) reported a strong correlation with a global composite similar to the PACC ( r = −0.53). Our study found that our mobile app-based tasks were significantly associated with the PACC5 ( r = 0.3–0.5), with similar effect sizes reported in previous work ( 15 , 19 , 34 ). However, our study did not use a composite measure but individual memory scores.…”

“…For example, automatically alternating test versions avoid or diminish practice effects ( 11 , 12 ) and enable more frequent and flexible cognitive testing ( 13 ). More frequent assessments may also increase reliability in cognitive measurement ( 14 , 15 ). Furthermore, it may yield more sensitive disease-related measures ( 16 , 17 ), such as accelerated forgetting ( 18 ), and the possibility of employing tests of long-term delayed recall.…”

“…However, several potential challenges remain unexplored as mobile app-based testing is performed in an uncontrolled environment ( 1 , 2 ). First, the feasibility of mobile app-based examinations in older adults, including examining the psychometric properties ( 15 , 19 – 21 ), participant experience ( 15 , 19 , 22 ), and external aspects of testing ( 16 ). Second, participant adherence to unsupervised and remote study designs is an important research question ( 23 ).…”