Training and Validation of Deep Learning-Based Auto-Segmentation Models for Lung Stereotactic Ablative Radiotherapy Using Retrospective Radiotherapy Planning Contours

Abstract

PurposeDeep learning-based auto-segmented contour (DC) models require high quality data for their development, and previous studies have typically used prospectively produced contours, which can be resource intensive and time consuming to obtain. The aim of this study was to investigate the feasibility of using retrospective peer-reviewed radiotherapy planning contours in the training and evaluation of DC models for lung stereotactic ablative radiotherapy (SABR).MethodsUsing commercial deep learning-based auto-segmentation software, DC models for lung SABR organs at risk (OAR) and gross tumor volume (GTV) were trained using a deep convolutional neural network and a median of 105 contours per structure model obtained from 160 publicly available CT scans and 50 peer-reviewed SABR planning 4D-CT scans from center A. DCs were generated for 50 additional planning CT scans from center A and 50 from center B, and compared with the clinical contours (CC) using the Dice Similarity Coefficient (DSC) and 95% Hausdorff distance (HD).ResultsComparing DCs to CCs, the mean DSC and 95% HD were 0.93 and 2.85mm for aorta, 0.81 and 3.32mm for esophagus, 0.95 and 5.09mm for heart, 0.98 and 2.99mm for bilateral lung, 0.52 and 7.08mm for bilateral brachial plexus, 0.82 and 4.23mm for proximal bronchial tree, 0.90 and 1.62mm for spinal cord, 0.91 and 2.27mm for trachea, and 0.71 and 5.23mm for GTV. DC to CC comparisons of center A and center B were similar for all OAR structures.ConclusionsThe DCs developed with retrospective peer-reviewed treatment contours approximated CCs for the majority of OARs, including on an external dataset. DCs for structures with more variability tended to be less accurate and likely require using a larger number of training cases or novel training approaches to improve performance. Developing DC models from existing radiotherapy planning contours appears feasible and warrants further clinical workflow testing.

Scalable radiotherapy data curation infrastructure for deep-learning based autosegmentation of organs-at-risk: A case study in head and neck cancer

“…Somewhat related, Wong et al described their RT patient selection and expert OAR segmentation efforts towards DL-based autosegmentation model evaluation, inclusive of capturing inter-expert variability (20). In a later report, the same institution described their expert OAR dataset preparation for model training and validation for a different anatomical site (21). The general sparsity of prior reporting may be indicative of a difficult peer review barrier (or perhaps a perception of a difficult peer review barrier)the potential challenges being that this and similar reports could be easily discarded as either non-hypothesis-driven or nonhypothesis-generating, or that they might be discounted under micro-dissection due to lack of novelty of constituent methods.…”

“…More recently, we have benefited from a virtual explosion in the application of deep convolutional neural networks (hereafter referred to as deep learning, or DL) for tackling complex imaging-related problems. In the context of RT, arguably the best-realized example of DL has been organ-at-risk (OAR) CT and MRI segmentation used as a starting point for the RT planning process (1,4,6,(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21). Currently, numerous vendors provide commercial autosegmentation solutions.…”

Clinical Validation of a Deep-Learning Segmentation Software in Head and Neck: An Early Analysis in a Developing Radiation Oncology Center

“…The clinical implementation of AC systems is providing benchmark data for the comparison and validation of such systems. Various studies have tested the potential role of AC software in different districts, obtaining results that seem to suggest the possibility of developing such systems in clinical practice [ 17 , 18 , 19 , 22 , 23 , 25 , 29 , 30 , 31 ].…”

“…Various experiences regarding clinical validation of AC models have already been reported in the literature, including testing AI-based delineation in different anatomical districts with single-institution workflow [ 17 ]. These trials tested AC models in multi-institutions validation groups, with the stated aim of demonstrating the validity of such systems in more complex RT treatment scenarios [ 18 , 19 ]. Furthermore, various studies have demonstrated that manual segmentation suffers from high inter-observers variability, and approaches to reduce such a source of uncertainty have been the subject of recent studies [ 20 , 21 ].…”

PSMA PET/CT as a predictive tool for subregional importance estimates in the parotid gland

“…Registered CT images were used for delineating parotid and submandibular glands. Limbus AI [25] was used for preliminary auto-segmentation of the glands, which were then manually refined by a single senior radiation oncologist, Jonn Wu.…”