Deep reinforcement learning identifies personalized intermittent androgen deprivation therapy for prostate cancer

Abstract

The evolution of drug resistance leads to treatment failure and tumor progression. Intermittent androgen deprivation therapy (IADT) helps responsive cancer cells compete with resistant cancer cells in intratumoral competition. However, conventional IADT is population-based and ignores the heterogeneous phenotypes of individual patients. To address this challenge, we developed a time-varied, mixed-effect, and generative Lotka-Volterra (tM-GLV) model to account for the heterogeneity of the evolution mechanism and the pharmacokinetics of individual patients. Then, we proposed a reinforcement learningenabled individualized IADT framework, namely, I2ADT, to learn the patient-specific tumor dynamics and derive the optimal drug administration policy. Experiments with clinical trial data demonstrated that the proposed I2ADT can significantly prolong the time to progression of prostate cancer patients with reduced cumulative drug dosage. This research elucidates the application of reinforcement learning techniques to identify personalized adaptive cancer therapy.

Learning to Adapt - Deep Reinforcement Learning in Treatment-Resistant Prostate Cancer

“…The approach we have presented here is not the first DRL framework to tackle adaptive therapy. The work of Lu et al [48] is focused solely on prostate cancer and used a different underlying model, but importantly trained the network on both the PSA as well as the senstitve and resistant populations; while PSA is easily measured in a blood draw there is no direct way to measure the numbers of sensitive and resistant cells in a real patient. While they were also able to produce significantly improved responses for a single patient, the generalisability of our approach, and the clear path for translation, make our implementation somewhat more robust.…”

“…Beyond uncertainty in the tumor's parameters, we cannot expect all tumors to behave in a way consistent with the virtual patient model (1), given the heterogeneity observed clinically in tumor dynamics and treatment response. We therefore considered the robustness of the DRL framework to variation in the underlying model and therefore tumor dynamics, by evaluating our framework on a modified Lotka-Volterra model introduced by Lu et al [48] and reproduced in Supplementary Information 5.2. This model uses exponential growth instead of logistic, such that the tumor may grow indefinitely, with a modified progression criterion based on the growth of the resistant subpopulation alone.…”

“…The patients who progress within 5000 days of AT are denoted by triangles. (b) Performance of a pre-trained DRL network when evaluated on a different tumor model (developed by Lu et al[48]) with modified dynamics (achieved through use of exponential growth terms, and where the progression criterion is based on the resistant subpopulation alone). A DRL framework evaluating treatment decisions weekly is able to consistently outperform AT.…”

Supplementary Data from Mathematical Model-Driven Deep Learning Enables Personalized Adaptive Therapy

“…Section 3.5 also explores the robustness of the DRL framework to variation in patient dynamics. Through this, we introduce a modified Lotka-Volterra model introduced by Lu et al [17], to demonstrate that a pre-trained DRL network can adapt to changes in the underlying tumor dynamics. Explicitly, this model may be written (in non-dimensional form) as:…”

“…For this model, progression was defined as growth in the resistant population alone to 0.1K R . The model was parameterized according to Table S2, replicating values used by Lu et al [17], and chosen to ensure that the profile in question does reach progression. Evaluating a pre-trained DRL framework on this new model, it attained a TTP of 1506 ± 3 days, outperforming the AT50 TTP of 1119 days (Figure S3).…”