Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge

Abstract

Mobile health, the collection of data using wearables and sensors, is a rapidly growing field in health research with many applications. Deriving validated measures of disease and severity that can be used clinically or as outcome measures in clinical trials, referred to as digital biomarkers, has proven difficult. In part due to the complicated analytical approaches necessary to develop these metrics . Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of Parkinson's Disease (PD) and severity of three PD symptoms: tremor, dyskinesia and bradykinesia. 40 teams from around the world submitted features, and achieved drastically improved predictive performance for PD (best AUROC=0.87), as well as severity of tremor (best AUPR=0.75), dyskinesia (best AUPR=0.48) and bradykinesia (best AUPR=0.95).

Developing better digital health measures of Parkinson’s disease using free living data and a crowdsourced data analysis challenge

“…Modeling strategies that take activity into account, for example human activity recognition (HAR) may be more successful in distinguishing movement types, though it is possible that certain types of activities will always be subject to high error rates in the prediction of dyskinesia. This is consistent with previous work that has shown good ability to predict symptom severity in the context of fixed activities 7,18 .…”

“…Consistent with previous efforts 7 , prediction of dyskinesia was more difficult than prediction of tremor or medication on/off state. This was supported by the fact that only 3 models significantly outperformed the null model for SC2 (dyskinesia), and of those, only one model's predictions significantly correlated with clinician ratings.…”

“…This is a necessary proof-of-concept toward the development and deployment of validated instruments for passive monitoring of PD. Past efforts have primarily focused on predicting symptom severity from short, well-defined tasks 5,7 . A few efforts have attempted to passively monitor PD symptoms in daily life, chiefly tremor and gait impairments 5,24 .…”

“…This investigation was carried out in two stages. In the first stage, several classes of heterogeneous ensemble methods [5][6][7][8] were tested in a nested cross-validation setup applied to the training sets of SC1-3 to determine the best ensemble method(s) for each subchallenge. The various ensemble methods showed variable performance across subchallenges when evaluated within the training data cross-validations (Supplementary Figure 5), though still outperforming the best individual team model in most cases.…”

Design of a virtual longitudinal observational study in Parkinson’s disease (AT‐HOME PD)

“…Consented participants enroll in the mPower Progression study, a remote study tracking the progression of PD using the mPower 2.0 smartphone application. The first version of the application was used by over 19,000 individuals with and without PD; the results of which have been previously published 34‐38 . The application includes active tasks of motor function (finger tapping, rest tremor, gait and balance assessed using embedded sensors in the smartphone) and optional passive monitoring (displacement tracking to capture a participant’s lifespace 39 (i.e., their mobility patterns) and accelerometer and gyroscope readings to measure gait during daily living) (Table 1).…”