Clinician versus machine: Reliability and responsiveness of motor endpoints in Parkinson's disease

Abstract

Background Enhancing the reliability and responsiveness of motor assessments required to demonstrate therapeutic efficacy is a priority for Parkinson's disease (PD) clinical trials. The objective of this study is to determine the reliability and responsiveness of a portable kinematic system for quantifying PD motor deficits as compared to clinical ratings. Methods Eighteen PD patients with subthalamic nucleus deep brain stimulation (DBS) performed three tasks for evaluating of resting tremor, postural tremor, and finger-tapping speed, amplitude, and rhythm while wearing a wireless motion-sensor unit (Kinesia) on the more-affected index finger. These tasks were repeated three times with DBS turned off and at each of 10 different stimulation amplitudes chosen to yield small changes in treatment response. Each task performance was video-recorded for subsequent clinician rating in blinded, randomized order. Test-retest reliability was calculated as intraclass correlation (ICC) and sensitivity was calculated as minimal detectable change (MDC) for each DBS amplitude. Results ICCs for Kinesia were significantly higher than those for clinician ratings of finger-tapping speed (p<0.0001), amplitude (p<0.0001), and rhythm (p<0.05), but were not significantly different for evaluations of resting or postural tremor. Similarly, Kinesia scores yielded a lower MDC as compared with clinician scores across all finger-tapping subscores (p<0.0001), but did not differ significantly for resting and postural tremor. Conclusions The Kinesia portable kinematic system can provide greater test-retest reliability and sensitivity to change than conventional clinical ratings for measuring bradykinesia, hypokinesia, and dysrhythmia in PD patients.

Tablet-Based Application for Objective Measurement of Motor Fluctuations in Parkinson Disease

“…These differences may have contributed to what represents only a moderate correlation with MDS-UPDRS-III scores. As subjective scales are limited in accuracy and reliability [1], a moderate correlation suggests a tablet-based application may be capable of detecting differences in motor function beyond those probed by the MDS-UPDRS-III. In fact, the test-retest reliability for both medication states was higher than that reported for the MDS-UPDRS-III (0.60–0.71) [26].…”

“…PD-specific motor scales also have limitations in logistic deployment, reliability, and spatial resolution [1, 2, 27]. These shortcomings may lower the signal-to-noise ratio in the measurement of relevant endpoints in clinical trials, lowering the ability of finding statistically significant differences between interventions of interest.…”

“…A growing body of evidence suggests that biometric monitoring using technology-based objective measures will become a valuable supplement to routine neurological evaluations [1-6]. Technology-based objective measures can reduce rater bias and interrater variability and increase sensitivity, elucidating important subclinical changes [4, 7, 8].…”

The First Frontier: Digital Biomarkers for Neurodegenerative Disorders

“…However, early studies suggest that a smartphone application in Parkinson disease can differentiate those with the disease from those without, be used to predict disease severity as assessed by traditional measures, and detect pharmacological effects of approved therapies [31]. In addition, where evidence of value has been generated [32-36], the US Food and Drug Administration has cleared some digital biomarkers for use [37, 38]. …”

Challenges in detecting disease modification in Parkinson's disease clinical trials

“…Studies have shown that this technique is feasible and can offer similar benefits to that of in-person evaluations [92,93]. Although many of these devices compare favourably with the "gold standard" of assessments of motor function, the MDS-UPDRS, and can provide greater test-retest reliability and sensitivity to change than conventional clinical ratings [94], there are current limitations to such approaches. Some aspects of assessment such as rigidity are extremely difficult to assess remotely and these devices remain to be validated in large population of patients with PD.…”