Automatic extraction of upper-limb kinematic activity using deep learning-based markerless tracking during deep brain stimulation implantation for Parkinson’s disease: A proof of concept study

Abstract

Optimal placement of deep brain stimulation (DBS) therapy for treating movement disorders routinely relies on intraoperative motor testing for target determination. However, in current practice, motor testing relies on subjective interpretation and correlation of motor and neural information. Recent advances in computer vision could improve assessment accuracy. We describe our application of deep learning-based computer vision to conduct markerless tracking for measuring motor behaviors of patients undergoing DBS surgery for the treatment of Parkinson’s disease. Video recordings were acquired during intraoperative kinematic testing (N = 5 patients), as part of standard of care for accurate implantation of the DBS electrode. Kinematic data were extracted from videos post-hoc using the Python-based computer vision suite DeepLabCut. Both manual and automated (80.00% accuracy) approaches were used to extract kinematic episodes from threshold derived kinematic fluctuations. Active motor epochs were compressed by modeling upper limb deflections with a parabolic fit. A semi-supervised classification model, support vector machine (SVM), trained on the parameters defined by the parabolic fit reliably predicted movement type. Across all cases, tracking was well calibrated (i.e., reprojection pixel errors 0.016–0.041; accuracies >95%). SVM predicted classification demonstrated high accuracy (85.70%) including for two common upper limb movements, arm chain pulls (92.30%) and hand clenches (76.20%), with accuracy validated using a leave-one-out process for each patient. These results demonstrate successful capture and categorization of motor behaviors critical for assessing the optimal brain target for DBS surgery. Conventional motor testing procedures have proven informative and contributory to targeting but have largely remained subjective and inaccessible to non-Western and rural DBS centers with limited resources. This approach could automate the process and improve accuracy for neuro-motor mapping, to improve surgical targeting, optimize DBS therapy, provide accessible avenues for neuro-motor mapping and DBS implantation, and advance our understanding of the function of different brain areas.

Smartphone video nystagmography using convolutional neural networks: ConVNG

“…With the introduction of DeepLabCut (DLC) [23] (and comparable tools, see [24] for a review), which is a highly accessible supervised deep learning framework extensively validated across several species [25], markerless pose tracking has now become the state-of-the-art in experimental neurosciences. In recent years, CV techniques have increasingly permeated into the realms of clinical and especially motor neuroscience [26][27][28][29][30][31][32][33]. At the core of these supervised DL frameworks commonly lie convolutional neural networks (CNNs) pretrained on several thousands to a million naturalistic images [34], which are fine-tuned by the user for specific use cases.…”

Anti-drift pose tracker (ADPT): A transformer-based network for robust animal pose estimation cross-species

“…The second strategy is signal processing filters such as median filter and low pass filter Stenum et al (2021 ); Pereira et al (2019 ); Luxem et al (2022 ); Weinreb et al (2023 ); Han et al (2023a ); Li and Lee (2021 ). They can efficiently remove most of the drifted points without human intervention, but they will also remove the subtle behaviors with high-frequency features such as self-grooming in autism mouse models Huang et al (2021 ) or tremor in animal models of Parkinson’s disease Baker et al (2022 ). The third strategy is fitting the drifted frames using linear dynamic models such as Keypoint-Moseq Weinreb et al (2023 ) and adaptive Kalman filter Huang et al (2022 ).…”

Phenotypic analysis of ataxia in spinocerebellar ataxia type 6 mice using DeepLabCut

“…like ataxia in mice and men [12][13][14][15] . For example, a recent study applied DLC to identify tremors and abnormal ataxic behaviors in an ataxic and tremor rodent model called the shaker and a mouse model for Spinocerebellar ataxia type 3 (SCA3) 12 .…”

“…Additionally, another group used the machine learning-based system JAABA (Janelia Atomatic Animal Behavior Annotator) to characterize the ataxic phenotype in a Purkinje cell-specific knockout of calcium/calmodulin-activated protein-phosphatase-2B (PP2B) mouse model for ataxia 16 . A more recent study in Parkinson's patients utilized DLC and algorithms to objectively correlate neural signals with movement to ideally place deep brain stimulation electrodes 13 .…”