A data-driven framework for selecting and validating digital health metrics: use-case in neurological sensorimotor impairments

Abstract

Digital health metrics promise to advance the understanding of impaired body functions, for example in neurological disorders. However, their clinical integration is challenged by an insufficient validation of the many existing and often abstract metrics. Here, we propose a data-driven framework to select and validate a clinically relevant core set of digital health metrics extracted from a technology-aided assessment. As an exemplary use-case, the framework is applied to the Virtual Peg Insertion Test (VPIT), a technology-aided assessment of upper limb sensorimotor impairments. The framework builds on a use-case-specific pathophysiological motivation of metrics, models demographic confounds, and evaluates the most important clinimetric properties (discriminant validity, structural validity, reliability, measurement error, learning effects). Applied to 77 metrics of the VPIT collected from 120 neurologically intact and 89 affected individuals, the framework allowed selecting 10 clinically relevant core metrics. These assessed the severity of multiple sensorimotor impairments in a valid, reliable, and informative manner. These metrics provided added clinical value by detecting impairments in neurological subjects that did not show any deficits according to conventional scales, and by covering sensorimotor impairments of the arm and hand with a single assessment. The proposed framework provides a transparent, step-by-step selection procedure based on clinically relevant evidence. This creates an interesting alternative to established selection algorithms that optimize mathematical loss functions and are not always intuitive to retrace. This could help addressing the insufficient clinical integration of digital health metrics. For the VPIT, it allowed establishing validated core metrics, paving the way for their integration into neurorehabilitation trials.

Assessment of Upper Limb Movement Impairments after Stroke Using Wearable Inertial Sensing

“…No clear correlations were found between the kinematic metrics and the FMA-UE, whereas trunk compensation and elbow flexion/extension showed strong correlation with the FMA-UE arm subsection, as well as the correlation between the FMA-UE hand subsection and finger flexion/extension. These findings are in line with existing research [ 24 , 44 ] and support the fact that kinematic parameters are, rather, complementary than redundant to standard clinical scales and potentially add clinically relevant information. The large interquartile ranges in all measured DOF in all study subjects illustrates the large variability in movement execution especially in non-cyclical discrete motions.…”

“…Being able to detect the main aspects of movement quality and impairments allows selecting and monitoring changes in functional outcome and planning interventions that target these aspects. Future research should consider and re-evaluate the outcome features and task considerations presented herein on larger sample sizes to further underpin existing evidence of sufficient validity and reliability for metrics of joint range of motion and trunk displacement [ 24 , 44 ]. Furthermore, analysis of the assessments’ clinimetric properties should be extended to domains sensitivity and specificity for differentiation physiological and pathological movement behavior.…”

Technology-aided assessment of functionally relevant sensorimotor impairments in arm and hand of post-stroke individuals

“…Recently, a core set of 10 kinematic and kinetic VPIT metrics was selected from a set of 77 candidate metrics based on an automated, data-driven metric selection process that optimizes clinically-relevant statistical criteria for longitudinally assessing impairments [ 32 ]. These metrics are extracted through an advanced processing and normalization pipeline that is applied to the position and grip force data from the VPIT, sampled at 1 kHz [ 32 ].…”

“…The Virtual Peg Insertion Test (VPIT) addresses many of the limitations of existing technology-aided assessments by recording movement and grip force patterns during a virtual goal-directed manipulation task requiring coordinated arm and hand movements [ 31 , 32 ]. Previous research indicated the feasibility of the approach in neurologic individuals with mild to moderate sensorimotor impairments [ 32 – 35 ]. In addition, ten digital health metrics capturing sensorimotor impairments have been established for the VPIT and allowed for an accurate discrimination between neurologically intact and affected individuals [ 32 ].…”

“…Previous research indicated the feasibility of the approach in neurologic individuals with mild to moderate sensorimotor impairments [ 32 – 35 ]. In addition, ten digital health metrics capturing sensorimotor impairments have been established for the VPIT and allowed for an accurate discrimination between neurologically intact and affected individuals [ 32 ]. However, whether the VPIT metrics provide a multi-dimensional evaluation of impairments in arm and hand that are functionally relevant has not been evaluated yet.…”