An Evaluation of KELVIN, an Artificial Intelligence Platform, as an Objective Assessment of the MDS UPDRS Part III

Abstract

Background: Parkinson’s disease severity is typically measured using the Movement Disorder Society Unified Parkinson’s disease rating scale (MDS-UPDRS). While training for this scale exists, users may vary in how they score a patient with the consequence of intra-rater and inter-rater variability. Objective: In this study we explored the consistency of an artificial intelligence platform compared with traditional clinical scoring in the assessment of motor severity in PD. Methods: Twenty-two PD patients underwent simultaneous MDS-UPDRS scoring by two experienced MDS-UPDRS raters and the two sets of accompanying video footage were also scored by an artificial intelligence video analysis platform known as KELVIN. Results: KELVIN was able to produce a summary score for 7 MDS-UPDRS part 3 items with good inter-rater reliability (Intraclass Correlation Coefficient (ICC) 0.80 in the OFF-medication state, ICC 0.73 in the ON-medication state). Clinician scores had exceptionally high levels of inter-rater reliability in both the OFF (0.99) and ON (0.94) medication conditions (possibly reflecting the highly experienced team). There was an ICC of 0.84 in the OFF-medication state and 0.31 in the ON-medication state between the mean Clinician and mean Kelvin scores for the equivalent 7 motor items, possibly due to dyskinesia impacting on the KELVIN scores. Conclusion: We conclude that KELVIN may prove useful in the capture and scoring of multiple items of MDS-UPDRS part 3 with levels of consistency not far short of that achieved by experienced MDS-UPDRS clinical raters, and is worthy of further investigation.

Computer vision quantification of whole-body Parkinsonian bradykinesia using a large multi-site population

“…The lower level of agreement for this site can be explained by a limitation of the model in lower severity patients: MDS-UPDRS clinical rating of 0 can only be either matched or overestimated by the model creating a tendency for an overestimation in the composite score. However, other causes are possible such as higher levels of clinician ratings variability for on-medication patients 23 or increased rater variability due to the non-clinical nature of the site 24 .…”

Epidemiology and Rating Scales

“…The total predicted score was calculated for each group and compared with the real group score and real total motor score. Inter-rater ICC (calculated with data from [8]) was used as gold standard performance, while a random imputation of scores was used as a control.…”

“…Background: Filipino's are the second largest sub-group of Asian Americans in the US and have greater socioeconomic and health disparities than many other Asian sub-groups. [1][2][3][4][5][6][7][8] In PD, evidence suggests that minorities and socioeconomically disadvantaged groups may have worse outcomes. [9][10][11] Compared to Chinese and Japanese PD patients, little is known about PD in the Filipino population.…”

“…Methods: We conducted a comprehensive literature review in PubMed and EMBASE of English language studies with age-stratified estimates published from January 2000 to March 2022. From 652 identified publications, 16 studies from 12 countries were included in our analysis [5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. Age categories varied from study to study.…”

An Artificial Intelligence Video-Based Assessment of Upper-limb Bradykinesia in Parkinson’s Disease

“…Many studies have proposed different approaches for analyzing videos of the Finger Tapping Testa motor examination part of the MDS-UPDRS that is used to assess upper limb bradykinesia by clinicians by evaluation movement aspects such as rhythm, speed and amplitude on a 0 (normal) to 4 (severe) scaleusing hand pose estimation algorithms for markerless motion tracking (H. Li et al, 2021;Z. Li et al, 2022;Monje et al, 2021;Morinan et al, 2023;K. Sibley et al, 2022;Vignoud et al, 2022;N.…”