Adaptive multi symptoms control of Parkinson's disease by deep reinforcement learning

Faraji, Behnam; Rouhollahi, Korosh; Paghaleh, Saeed Mollahoseini; Gheisarnejad, Meysam; Khooban, Mohammad‐Hassan

doi:10.1016/j.bspc.2022.104410

Cited by 8 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, where δ(i) = Q(s, a) − (r + γQ(s ′ , a ′ )) (10) In the equation the level of prioritization is governed by the hyperparameter α.…”

Section: Td7mentioning

confidence: 99%

“…While there exists no treatment for the underlying root of neurodegenerative diseases, several treatment options have been proposed to manage the symptoms and improve the patient's quality of life. These methods include invasive treatment options such as deep brain stimulation [9], [10], neurosurgery [11] and stem cell treatment methods [12]. However, these treatment options often come with severe side effects and high costs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning to Suppress Tremors: A Deep Reinforcement Learning-Enabled Soft Exoskeleton for Parkinson's Patients

Tamás,

Földi,

Makk

et al. 2024

Preprint

View full text Add to dashboard Cite

Neurological tremors, prevalent among a large population, are one of the most rampant movement disorders. While biomechanical loading and exoskeletons show promise in enhancing patient well-being, traditional control algorithms limit their efficacy in dynamic movements and personalized interventions. Furthermore, there exists a pressing need for more comprehensive and robust validation methods to ensure the effectiveness and generalizability of proposed solutions. This paper proposes a physical simulation approach modelling multiple arm joints and tremor propagation. This study also introduces a novel adaptable reinforcement learning environment tailored for disorders with tremors. Furthermore we present a deep reinforcement learning based encoder-actor controller for Parkinson's tremor present in various shoulder and elbow joint axes, displayed in dynamic movements. We found that the controller is capable of suppressing tremors in all recorded dynamic movements. The control achieves an average tremor of over 57.43% for three out of the four distinct movement trajectories. Notably, the controller attains a maximum accuracy of 99.49%. Our findings suggest that a deep reinforcement learning based control strategy offers a viable solution for tremor suppression in real-world scenarios. By overcoming the limitations of traditional control algorithms, this work takes a new step in the adaptation of biomechanical loading into the everyday life of patients. This work also opens avenues for more adaptive and personalized interventions in managing movement disorders.

show abstract

“…, where δ(i) = Q(s, a) − (r + γQ(s ′ , a ′ )) (10) In the equation the level of prioritization is governed by the hyperparameter α.…”

Section: Td7mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning to Suppress Tremors: A Deep Reinforcement Learning-Enabled Soft Exoskeleton for Parkinson's Patients

Tamás,

Földi,

Makk

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…While there exists no treatment for the underlying root of neurodegenerative diseases, several treatment options have been proposed to manage the symptoms and improve the patient's quality of life. These methods include invasive treatment options such as medication [9], deep brain stimulation [10], [11], neurosurgery [12], traditional Chinese treatments [13] and stem cell treatment methods [14]. However, these treatment options often come with severe side effects and high costs.…”

Section: Introductionmentioning

confidence: 99%

Learning to Suppress Tremors: A Deep Reinforcement Learning-Enabled Soft Exoskeleton for Parkinson's Patients

Tamás,

Földi,

Makk

et al. 2024

Preprint

View full text Add to dashboard Cite

Neurological tremors, prevalent among a large population, are one of the most rampant movement disorders. Biomechanical loading and the utilization of exoskeletons have been demonstrated to enhance the everyday life of patients. However the reliance on traditional control algorithms limit the use of this treatment in dynamic movements and in diseases where the model of the underlying condition is not available or where the model is susceptible to change. Here we present a novel deep reinforcement learning based control strategy capable of handling a wide range of tremors across multiple dynamic movements. Our approach incorporates the use of encoder networks, modified buffers and heavily shaped rewards to create actuation forces for the exoskeleton. We find that the control strategy is capable of efficiently suppressing tremors with different frequencies originating from multiple concurrent joint axes, present in a multitude of dynamic movements.

show abstract

“…Therefore, it is imperative to design a new DBS control method that can simultaneously address the dynamics, nonlinearity, and non-stationarity in the cortex-BG-thalamus network for PD. Recently, an adaptive PI controller has been proposed to address nonlinear neural dynamics in PD, but only tested to control limited target levels of beta oscillation power [37,38,39]. Also, it is not tested against non-stationary neural dynamics.…”

Section: Introductionmentioning

confidence: 99%

Robust Adaptive Deep Brain Stimulation Control of Non-Stationary Cortex-Basal Ganglia-Thalamus Network Models in Parkinson’s Disease

Fang,

Berman,

Wang

et al. 2023

Preprint

View full text Add to dashboard Cite

Closed-loop deep brain stimulation (DBS) is a promising therapy for Parkinson’s disease (PD) that works by adjusting DBS patterns in real time from the guidance of feedback neural activity. Current closed-loop DBS mainly uses threshold-crossing on-off controllers or linear time-invariant (LTI) controllers to regulate the basal ganglia (BG) beta band oscillation power. However, the critical cortex-BG-thalamus network dynamics underlying PD are nonlinear, non-stationary, and noisy, hindering the accurate and robust control of PD neural dynamics using current closed-loop DBS methods. Here, we develop a new robust adaptive closed-loop DBS method for regulating cortex-BG-thalamus network dynamics in PD. We first build an adaptive state-space model to quantify the dynamic, nonlinear, and non-stationary neural activity. We then construct an adaptive estimator to track the nonlinearity and non-stationarity in real time. We next design a robust controller to automatically determine the DBS frequency based on the estimated PD neural state while reducing the system’s sensitivity to high-frequency noise. We adopt and tune a biophysical cortex-BG-thalamus network model as a testbed to simulate various nonlinear and non-stationary neural dynamics for evaluating DBS methods. We find that under different nonlinear and non-stationary neural dynamics, our robust adaptive DBS method achieved accurate regulation of the BG beta band oscillation power with small control error, bias, and deviation. Moreover, the accurate regulation generalizes across different therapeutic targets and consistently outperforms state-of-the-art on-off and LTI DBS methods. These results have implications for future designs of clinically-viable closed-loop DBS systems to treat PD and other neurological and neuropsychiatric disorders.

show abstract

Adaptive multi symptoms control of Parkinson's disease by deep reinforcement learning

Cited by 8 publications

References 30 publications

Learning to Suppress Tremors: A Deep Reinforcement Learning-Enabled Soft Exoskeleton for Parkinson's Patients

Learning to Suppress Tremors: A Deep Reinforcement Learning-Enabled Soft Exoskeleton for Parkinson's Patients

Learning to Suppress Tremors: A Deep Reinforcement Learning-Enabled Soft Exoskeleton for Parkinson's Patients

Robust Adaptive Deep Brain Stimulation Control of Non-Stationary Cortex-Basal Ganglia-Thalamus Network Models in Parkinson’s Disease

Contact Info

Product

Resources

About