Developing better digital health measures of Parkinson’s disease using free living data and a crowdsourced data analysis challenge

Abstract

One of the promising opportunities of digital health is its potential to lead to more holistic understandings of diseases by interacting with the daily life of patients and through the collection of large amounts of real world data. Validating and benchmarking indicators of disease severity in the home setting is difficult, however, given the large number of confounders present in the real world and the challenges in collecting ground truth data in the home. Here we leverage two datasets with continuous wrist-worn accelerometer data coupled with frequent symptom reports in the home setting, to develop digital biomarkers of symptom severity. Using these data, we performed a public benchmarking challenge in which participants were asked to build measures of severity across 3 symptoms (on/off medication, dyskinesia, and tremor). 42 teams participated and performance was improved over baseline models for each subchallenge. Additional ensemble modeling across submissions further improved performance, and the top models validated in a subset of patients whose symptoms were observed and rated by trained clinicians.

Identification of kinematic biomarkers for self-reported Parkinson's disease symptoms

“…The session data les were standardized to a start time of 0 to remove time-of-day information as well as information about the temporal order of the sessions. Declaration of Helsinki (see also 34 ) .…”

“…Subsequently, a new challenge, the Biomarker and Endpoint Assessment to Track Parkinson's Disease (BEAT-PD) DREAM challenge, was launched to determine whether PD severity could be evaluated from passively collected kinematic data recorded using consumer wearables during the course of daily life. In this challenge, the participants were provided with raw kinematic data of PD patients recorded at home and were asked to predict the patients' self-reported medication state and symptom severity 34 . This article describes our top-scoring solution for this challenge.…”

Integrating multimodal data through interpretable heterogeneous ensembles

“…(2018); Stanescu and Pandey (2017)), enhancing the predictive power of DREAM Challenges (Sieberts et al . (2016, 2021)) and modeling infectious disease epidemics (Ray and Reich (2018)). However, these applications were limited to individual (unimodal) datasets.…”

“…Furthermore, homogeneous methods learn individual models as a part of the ensemble process, and thus cannot integrate models that have been derived independently a priori. The advantages of heterogeneous ensembles have been demonstrated in several biomedical applications, such as protein function prediction (Whalen et al (2016); Wang et al (2018); Stanescu and Pandey (2017)), enhancing the predictive power of DREAM Challenges (Sieberts et al (2016(Sieberts et al ( , 2021) and modeling infectious disease epidemics (Ray and Reich (2018)). However, these applications were limited to individual (unimodal) datasets.…”