Disease severity classification using passively collected smartphone-based keystroke dynamics within multiple sclerosis

Abstract

Multiple Sclerosis (MS) is a progressive demyelinating disease of the central nervous system characterised by a wide range of motor and non-motor symptoms. The level of disability of people with MS (pwMS) is based on a wide range of clinical measures, though their frequency of evaluation and inaccuracies coming from objective and self-reported evaluations limits these assessments. Alternatively, remote health monitoring through devices can offer a cost-efficient solution to gather more reliable, objective measures continuously. Measuring smartphone keyboard interactions is a promising tool since typing and, thus, keystroke dynamics are likely influenced by symptoms that pwMS can experience. Therefore, this paper aims to investigate whether keyboard interactions gathered on a person’s smartphone can provide insight into the clinical status of pwMS leveraging machine learning techniques. In total, 24 Healthy Controls (HC) and 102 pwMS were followed for one year. Next to continuous data generated via smartphone interactions, clinical outcome measures were collected and used as targets to train four independent multivariate binary classification pipelines in discerning pwMS versus HC and estimating the level of disease severity, manual dexterity and cognitive capabilities. The final models yielded an AUC-ROC in the hold-out set above 0.7, with the highest performance obtained in estimating the level of fine motor skills (AUC-ROC=0.753). These findings show that keyboard interactions combined with machine learning techniques can be used as an unobtrusive monitoring tool to estimate various levels of clinical disability in pwMS from daily activities and with a high frequency of sampling without increasing patient burden.

Identifying factors impacting missingness within smartphone-based research: Implications for intensive longitudinal studies of adolescent suicidal thoughts and behaviors.

“…Additionally, an analysis across several studies found that rates of missing GPS and accelerometer data did not differ by sex, education, or age; however, Black participants were missing more accelerometer data (Kiang et al, 2021). Notably, although missingness of keyboard input data (keystroke logging) has been documented across studies (Hoeijmakers et al, 2023; McNeilly et al, 2023), to our knowledge no study has examined mechanisms driving keyboard missingness. Overall, there are significant gaps in our understanding of whether psychological, sociodemographic, and study design factors are associated with missing data in passive smartphone sensing research, particularly among youth and clinical populations (Renn et al, 2018).…”

A Review on Smartphone Keystroke Dynamics as a Digital Biomarker for Understanding Neurocognitive Functioning

“…Using one year’s worth of data, researchers found that participants with MS who had worse arm motor function had a higher latency between keypresses, and participants with MS who had a decreased processing speed corresponded with a higher latency using punctuation and backspace keys [ 50 ]. Using the same dataset, researchers were also able to estimate the levels of disease severity, manual dexterity, and cognitive capabilities from keystroke dynamics using a machine learning model that used three predictors (a time-related cluster, a cognitive-related cluster, and the number of times autofill was used) [ 51 ]. Participants with MS who were quicker to correct and adjust their texting had higher SDMT scores, an indicator of cognitive functioning, which helped with model predictions [ 51 ].…”

“…Using the same dataset, researchers were also able to estimate the levels of disease severity, manual dexterity, and cognitive capabilities from keystroke dynamics using a machine learning model that used three predictors (a time-related cluster, a cognitive-related cluster, and the number of times autofill was used) [ 51 ]. Participants with MS who were quicker to correct and adjust their texting had higher SDMT scores, an indicator of cognitive functioning, which helped with model predictions [ 51 ]. These studies show that keystroke dynamics can be used as potential biomarkers for MS before significant disease onset, which would allow for earlier treatments and preventative care.…”

Tap and Swipe Smartphone Gestures Indicating Hand Tremor and Finger Joint Range in Motion

“…Smartphones for people with MS have been used to investigate fatigue using tapping gestures [17], dexterity through pinching motions [18], and general progression of the condition through non-invasive keyboard gestures [19]. To the best of the authors' knowledge, force control for people with MS has not been investigated using smartphone screen measurements.…”