Clinical evaluation of two AI models for automated breast cancer plan generation

Abstract

Background Artificial intelligence (AI) shows great potential to streamline the treatment planning process. However, its clinical adoption is slow due to the limited number of clinical evaluation studies and because often, the translation of the predicted dose distribution to a deliverable plan is lacking. This study evaluates two different, deliverable AI plans in terms of their clinical acceptability based on quantitative parameters and qualitative evaluation by four radiation oncologists. Methods For 20 left-sided node-negative breast cancer patients, treated with a prescribed dose of 40.05 Gy, using tangential beam intensity modulated radiotherapy, two model-based treatment plans were evaluated against the corresponding manual plan. The two models used were an in-house developed U-net model and a vendor-developed contextual atlas regression forest model (cARF). Radiation oncologists evaluated the clinical acceptability of each blinded plan and ranked plans according to preference. Furthermore, a comparison with the manual plan was made based on dose volume histogram parameters, clinical evaluation criteria and preparation time. Results The U-net model resulted in a higher average and maximum dose to the PTV (median difference 0.37 Gy and 0.47 Gy respectively) and a slightly higher mean heart dose (MHD) (0.01 Gy). The cARF model led to higher average and maximum doses to the PTV (0.30 and 0.39 Gy respectively) and a slightly higher MHD (0.02 Gy) and mean lung dose (MLD, 0.04 Gy). The maximum MHD/MLD difference was ≤ 0.5 Gy for both AI plans. Regardless of these dose differences, 90–95% of the AI plans were considered clinically acceptable versus 90% of the manual plans. Preferences varied between the radiation oncologists. Plan preparation time was comparable between the U-net model and the manual plan (287 s vs 253 s) while the cARF model took longer (471 s). When only considering user interaction, plan generation time was 121 s for the cARF model and 137 s for the U-net model. Conclusions Two AI models were used to generate deliverable plans for breast cancer patients, in a time-efficient manner, requiring minimal user interaction. Although the AI plans resulted in slightly higher doses overall, radiation oncologists considered 90–95% of the AI plans clinically acceptable.

Knowledge-based versus deep learning based treatment planning for breast radiotherapy

“…As our results show similar values for both DL and KBP solutions we wish to highlight the recommendation that departments should weigh differences as being clinically relevant or not. Concerning potential limitations, we acknowledge that the use of anisotropic analytical algorithm in our study versus the use of collapsed cone convolution in the works of Kneepkens et al [26] may cause differences in the results. However, our comparison replicates a daily situation with commercially available software in use.…”

Clinical evaluation of a deep learning segmentation model including manual adjustments afterwards for locally advanced breast cancer

“…In a future study, the dosimetric impact of this auto-segmentation model could be evaluated. In order to perform such an analysis without any interobserver bias, ideally this should be done by the use of automatically generated treatments plans [21] , [22] . Moreover, these plans should be based on predefined clinical goals which are widely accepted to make such a comparison more generally useful [23] .…”

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings

“…Few studies used randomised trial designs, opting instead for designs such as weaker historical case controls. Accuracy was most commonly measured, although a few studies examined safety and clinician time (e.g., [44,46,53]). Effects on care-delivery were assessed using a variety of measures including time to treatment (e.g., [29,39,67,68]).…”

“…For prostate radiotherapy, Cha et al [45] demonstrated clinical utility of AI for MRbased planning with 65% of cases requiring no more than minor edits, and a time saving of 12 min (30% of total contouring time) for physicians. Kneepkens et al [46] found that although automatically generated plans resulted in slightly higher doses, they were clinically acceptable (AI: 90-95% vs. manual: 90%) and time-efficient.…”