Naturalistic smartphone keyboard typing reflects processing speed and executive function

Abstract

Objective:The increase in smartphone usage has enabled the possibility of more accessible ways to conduct neuropsychological evaluations. The objective of this study was to determine the feasibility of using smartphone typing dynamics with mood scores to supplement cognitive assessment through trail making tests. Methods: Using a custom-built keyboard, naturalistic keypress dynamics were unobtrusively recorded in individuals with bipolar disorder (n = 11) and nonbipolar controls (n = 8) on an Android smartphone. Keypresses were matched to digital trail making tests part B (dTMT-B) administered daily in two periods and weekly mood assessments. Following comparison of dTMT-Bs to the pencil-and-paper equivalent, longitudinal mixed-effects models were used to analyze daily dTMT-B performance as a function of typing and mood.Results: Comparison of the first dTMT-B to paper TMT-B showed adequate reliability (intraclass correlations = 0.74). In our model, we observed that participants who typed slower took longer to complete dTMT-B (b = 0.189, p < .001). This trend was also seen in individual fluctuations in typing speed and dTMT-B performance (b = 0.032, p = .004). Moreover, participants who were more depressed completed the dTMT-B slower than less depressed participants (b = 0.189, p < .001). A practice effect was observed for the dTMT-Bs. Conclusion:Typing speed in combination with depression scores has the potential to infer aspects of cognition (visual attention, processing speed, and task switching) in people's natural environment to complement formal in-person neuropsychological assessments that commonly include the trail making test.

Diagnostic accuracy of keystroke dynamics as digital biomarkers for fine motor decline in neuropsychiatric disorders: a systematic review and meta-analysis

“…62002198, No. 61902208 Goni et al (2021) 42 PD In-the-wild (NR) 970 (59.85, 9.05, 35%) 1630 (46.84, 10.05, 15.2%) Clinical assessment Smartphone application with 4 tasks: gait, balance, voice and tapping Subject level 700 features extracted, comprising statistical features of time and frequency locomotion NA Least absolute shrinkage and selection operator (LASSO), RF, SVM NR Surangsrirat et al (2022) 41 PD In-the-wild (NR) 1851 (44.27, 0.44, 31.5%) NA Self-reports Demographics, MDS-UPDRS I–II, PDQ-8, memory, tapping, voice, and walking Subject level High and low order statistics of keystroke dynamics NA K-means unsupervised clustering National Science and Technology Development Agency (NSTDA), Thailand Zulueta et al (2021) 62 Bipolar disorder In-the-wild (35 months) 227 (35, 11, 75%) 117 (41, 16, 60%) Self-reports Keystroke dynamics and typing metadata (autocorrect and backspace rate) Session level Low order statistics of keystroke dynamics, entropy (complexity) features NA RF Mood Challenge for Research kit 1R01MH120168 Ross et al (2021) 61 Bipolar disorder In-the-wild (2 months) 11 (47, 10.6, 72.7%) 8 (46.1, 10.6, 62.5%) Hybrid (clinical assessment and self-reports) Keystroke timing data Session level Low-order statistics Longitudinal mixed effects NA T...…”

“…In this vein, the analysis of typing kinetics, along with the clinical scores, facilitated the prediction of brain age and revealed that the predicted age of bipolar disorders patients is higher than their actual age, compared to healthy controls, reflecting a marker of brain pathology 62 . Moreover, keystroke dynamics predict cognitive decline, diminished visual attention, reduced processing speed and task switching in bipolar disorder patients 61 .…”

Capturing Cognitive Aging in Vivo: Application of a Neuropsychological Framework for Emerging Digital Tools

“…Of note, these studies were exploratory in nature and lacked a priori hypotheses to guide analyses. A separate pilot study of adults with and without bipolar disorder examined performance on a digital trail making test and found associations between smartphone typing speed and typing speed variability and test performance, suggesting a possible link between executive functioning and keystroke measures [ 29 ]. In the context of MCI and dementia, a feasibility study employing multiple sensor streams and machine learning models identified 5 digital features that discriminated symptomatic (MCI, mild AD) from asymptomatic groups; these features included typing speed, regularity in behavior (via first and last phone use), number of received text messages, reliance on helper apps, and survey compliance [ 55 ].…”

“…The existing literature is less clear on patterns of variability in the transition from MCI to dementia [ 123 ], and we acknowledge the possibility that for relatively simple activities that individuals with mild dementia still perform (eg, movement trajectories within the home, incoming phone calls, sleep/wake cycle), variability may continue to increase in the mild dementia stage followed by eventual decline as abilities further decline. Thus, model predictions should be tested and interpreted with attention to task demands, as well as other contextual features, including the time of day [ 29 ], mood, and technology use habits. In other words, the progression from increased variability to decreased variability and complete failure to act depicted in Figure 1 is expected with increasing severity of impairment, though impairment level is determined by more than just clinical status.…”

“…New mobile and computerized platforms with enhanced efficiency and sensitivity, such as repeat ambulatory cognitive assessments, address some of these methodological drawbacks [25] and have been examined in various studies among populations comprising healthy and community-based older adults [22,26], those with preclinical AD [27], and those with MCI or early dementia [28]. However, many of these methods continue to be (A) modeled after traditional tests that measure isolated cognitive domains with limited ecological validity; (B) susceptible to practice effects [29]; (C) influenced by socioeconomic status and cultural factors [30][31][32]; and (D) prone to challenges with adherence even among highly motivated and engaged individuals, particularly with longer study durations [27,33,34]. Thus, while tremendous advances have been made in the realm of digital cognitive assessment, existing methods continue to show limited generalizability to diverse populations and real-world settings, even when used at home outside of the clinic.…”