Clinical assessments of movement disorders currently rely on the administration of rating scales, which, while clinimetrically validated and reliable, rely on clinicians’ subjective analyses, resulting in interrater differences. Intraoperative microelectrode recording for deep brain stimulation targeting similarly relies on clinicians’ subjective evaluations of movement-related neural activity. Digital motion tracking can improve the diagnosis, assessment, and treatment of movement disorders by generating objective, standardized measures of patients’ kinematics. Motion tracking with concurrent neural recording also enables motor neuroscience studies to elucidate the neurophysiology underlying movements. Despite these promises, motion tracking has seen limited adoption in clinical settings due to the drawbacks of conventional motion tracking systems and practical limitations associated with clinical settings. However, recent advances in deep learning based computer vision algorithms have made accurate, robust markerless motion. tracking viable in any setting where digital video can be captured. Here, we review and discuss the potential clinical applications and technical limitations of deep learning based markerless motion tracking methods with a focus on DeepLabCut (DLC), an open-source software package that has been extensively applied in animal neuroscience research. We first provide a general overview of DLC, discuss its present usage, and describe the advantages that DLC confers over other motion tracking methods for clinical use. We then present our preliminary results from three ongoing studies that demonstrate the use of DLC for 1) movement disorder patient assessment and diagnosis, 2) intraoperative motor mapping for deep brain stimulation targeting and 3) intraoperative neural and kinematic recording for basic human motor neuroscience.

Optimal placement of deep brain stimulation (DBS) therapy for treating movement disorders routinely relies on intraoperative motor testing for target determination. However, in current practice, motor testing relies on subjective interpretation and correlation of motor and neural information. Recent advances in computer vision could improve assessment accuracy. We describe our application of deep learning-based computer vision to conduct markerless tracking for measuring motor behaviors of patients undergoing DBS surgery for the treatment of Parkinson’s disease. Video recordings were acquired during intraoperative kinematic testing (N = 5 patients), as part of standard of care for accurate implantation of the DBS electrode. Kinematic data were extracted from videos post-hoc using the Python-based computer vision suite DeepLabCut. Both manual and automated (80.00% accuracy) approaches were used to extract kinematic episodes from threshold derived kinematic fluctuations. Active motor epochs were compressed by modeling upper limb deflections with a parabolic fit. A semi-supervised classification model, support vector machine (SVM), trained on the parameters defined by the parabolic fit reliably predicted movement type. Across all cases, tracking was well calibrated (i.e., reprojection pixel errors 0.016–0.041; accuracies >95%). SVM predicted classification demonstrated high accuracy (85.70%) including for two common upper limb movements, arm chain pulls (92.30%) and hand clenches (76.20%), with accuracy validated using a leave-one-out process for each patient. These results demonstrate successful capture and categorization of motor behaviors critical for assessing the optimal brain target for DBS surgery. Conventional motor testing procedures have proven informative and contributory to targeting but have largely remained subjective and inaccessible to non-Western and rural DBS centers with limited resources. This approach could automate the process and improve accuracy for neuro-motor mapping, to improve surgical targeting, optimize DBS therapy, provide accessible avenues for neuro-motor mapping and DBS implantation, and advance our understanding of the function of different brain areas.

No abstract

The expanding application of deep brain stimulation (DBS) therapy both drives and is informed by our growing understanding of disease pathophysiology and innovations in neurosurgical care. Neurophysiological targeting, a mainstay for identifying optimal, motor responsive targets, has remained largely unchanged for decades. Utilizing deep learning-based computer vision and related computational methods, we developed an effective and simple intraoperative approach to objectively correlate neural signals with movements, automating and standardizing the otherwise manual and subjective process of identifying ideal DBS electrode placements. Kinematics are extracted from video recordings of intraoperative motor testing using a trained deep neural network and compared to multi-unit activity recorded from the subthalamic nucleus. Neuro-motor correlations were quantified using dynamic time warping with the strength of a given comparison measured by comparing against a null distribution composed of related neuro-motor correlations. This objective measure was then compared to clinical determinations as recorded in surgical case notes. In seven DBS cases for treatment of Parkinson’s disease, 100 distinct motor testing epochs were extracted for which clear clinical determinations were made. Neuro-motor correlations derived by our automated system compared favorably with expert clinical decision making in post-hoc comparisons, although follow-up studies are necessary to determine if improved correlation detection leads to improved outcomes. By improving the classification of neuro-motor relationships, the automated system we have developed will enable clinicians to maximize the therapeutic impact of DBS while also providing avenues for improving continued care of treated patients.