Intelligent inverse treatment planning via deep reinforcement learning, a proof-of-principle study in high dose-rate brachytherapy for cervical cancer

Abstract

Inverse treatment planning in radiation therapy is formulated as solving optimization problems. The objective function and constraints consist of multiple terms designed for different clinical and practical considerations. Weighting factors of these terms are needed to define the optimization problem. While a treatment planning optimization engine can solve the optimization problem with given weights, adjusting the weights to yield a high-quality plan is typically performed by a human planner. Yet the weight-tuning task is labor intensive, time consuming, and it critically affects the final plan quality. An automatic weight-tuning approach is strongly desired. The procedure of weight adjustment to improve the plan quality is essentially a decision-making problem. Motivated by the tremendous success in deep learning for decision making with human-level intelligence, we propose a novel framework to adjust the weights in a human-like manner. This study uses inverse treatment planning in high-dose-rate brachytherapy (HDRBT) for cervical cancer as an example. We develop a weight-tuning policy network (WTPN) that observes dose volume histograms of a plan and outputs an action to adjust organ weighting factors, similar to the behaviors of a human planner. We train the WTPN via end-to-end deep reinforcement learning. Experience replay is performed with the epsilon greedy algorithm. After training is completed, we apply the trained WTPN to guide treatment planning of five testing patient cases. It is found that the trained WTPN successfully learns the treatment planning goals and is able to guide the weight tuning process. On average, the quality score of plans generated under the WTPN's guidance is improved by ~8.5% compared to the initial plan with arbitrarily set weights, and by 10.7% compared to the plans generated by human planners. To our knowledge, this is the first time that a tool is developed to adjust organ weights for the treatment planning optimization problem in a human-like fashion based on intelligence learnt from a training process. This is different from existing strategies based on pre-defined rules. The study demonstrates potential feasibility to develop intelligent treatment planning approaches via deep reinforcement learning.

A preliminary study on a multiresolution‐level inverse planning approach for Gamma Knife radiosurgery

“…Physicians just need to select the plan that has the preferred trade‐off. Another way is to use artificial intelligence to build and train a virtual planner to develop a human‐level intelligence on how to automatically adjust the priorities based on the intermediate plan quality . Our future work is to take the latter way to learn the physicians’ preferred trade‐offs from previously treated patient cases and automate the fine‐tuning process.…”

A GPU-based multi-criteria optimization algorithm for HDR brachytherapy

“…In MCO, a plan pool is constructed by generating plans with various trade-offs on Pareto surfaces (Craft et al 2006, Teichert et al 2011. Similar studies can also be found in brachytherapy (van der Meer et al 2018, Shen et al 2018, Zhou et al 2017, Cui et al 2018a, Cui et al 2018b.…”

In vivo dosimetry in brachytherapy: Requirements and future directions for research, development, and clinical practice

“…), TCP/NTCP models to estimate the clinical effect of a dose deviation, and tools to pinpoint the its most likely cause. Such tools could employ artificial intelligence as already evaluated for treatment planning optimization [64] and inter-fraction adaptation [65] .…”