Quantum deep reinforcement learning for clinical decision support in oncology: application to adaptive radiotherapy

Abstract

Subtle differences in a patient’s genetics and physiology may alter radiotherapy (RT) treatment responses, motivating the need for a more personalized treatment plan. Accordingly, we have developed a novel quantum deep reinforcement learning (qDRL) framework for clinical decision support that can estimate an individual patient’s dose response mid-treatment and recommend an optimal dose adjustment. Our framework considers patients’ specific information including biological, physical, genetic, clinical, and dosimetric factors. Recognizing that physicians must make decisions amidst uncertainty in RT treatment outcomes, we employed indeterministic quantum states to represent human decision making in a real-life scenario. We paired quantum decision states with a model-based deep q-learning algorithm to optimize the clinical decision-making process in RT. We trained our proposed qDRL framework on an institutional dataset of 67 stage III non-small cell lung cancer (NSCLC) patients treated on prospective adaptive protocols and independently validated our framework in an external multi-institutional dataset of 174 NSCLC patients. For a comprehensive evaluation, we compared three frameworks: DRL, qDRL trained in a Qiskit quantum computing simulator, and qDRL trained in an IBM quantum computer. Two metrics were considered to evaluate our framework: (1) similarity score, defined as the root mean square error between retrospective clinical decisions and the AI recommendations, and (2) self-evaluation scheme that compares retrospective clinical decisions and AI recommendations based on the improvement in the observed clinical outcomes. Our analysis shows that our framework, which takes into consideration individual patient dose response in its decision-making, can potentially improve clinical RT decision-making by at least about 10% compared to unaided clinical practice. Further validation of our novel quantitative approach in a prospective study will provide a necessary framework for improving the standard of care in personalized RT.

ARCliDS: A Clinical Decision Support System for AI-assisted Decision-Making in Response-Adaptive Radiotherapy

“…ARCliDS presents a significant improvement to Tseng et al’s 17 and Niraula et al’s 5 methods. The improvement comes from the graphical representation of patients’ features.…”

“…To extend the sample size of our dataset, we generated 10,000 synthetic patients via Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) 4,5 . We compared the distribution of synthetic patient with the original patient population data using the Jensen Shannon Divergence (JSD) metric as shown in the subsequent sections.…”

“…We have proposed a linear-quadratic-linear (LQL) type TF. This is an improvement from Niraula et al’s 5 work which employed a linear-quadratic (LQ) type function. LQL model is a generalization of LQ model that covers both RT and SBRT.…”

“…LQL model is a generalization of LQ model that covers both RT and SBRT. Furthermore, for the RTOE, we applied GNN to general logistic function guided double NN architecture, which was developed by Niraula et al 5 To improve the robustness of ODM, we have replaced the online learning approach with the Planning and Learning approach 21 . We trained and validated ARCliDS in two different cancer sites and different treatment types to show its versatility.…”

A Survey on Quantum Reinforcement Learning

“…Finally, Refs. [Li+20a;Nir+21] apply QiRL to human decision making behavior. Recently, the quantum-inspired approach to action selection in RL was transferred to experience replay buffer sampling in Q-learning [Wei+22].…”